Color is iffy for wood ID purposes on all but recently-worked wood surfaces, as wood color changes with time even in the absence of any finish (and even when fresh, natural variation throws a wrench to the works). What doesn’t change, regardless of fit, finish or individual tree quirks, is wood structure, which remains"readable" even in charcoal.
When I look at a piece of wood, the first thing I check is to see if the wood is ring-porous, diffuse-porous or coniferous. This is readily evident without magnification, and instantly narrows down the options. The ring-porous oak, ash and elm stand out from diffuse-porous maple, birch, cherry, alder etc. at a glance. The conifers, lacking vessels (which make the pores of hardwoods) are also a distinctive group with a close look.
Next, with hardwoods, is the presence or absence of (visible-to-naked-eye) rays, and the size, shape and pattern of them. Woods like oak, maple and beech are most easily ID’d by their distinctive rays. Ring-porous oak and elm both have visible rays, but the rays are clearly different from each other, as are the rays of diffuse-porous maple and beech.
The pattern of knots, when present, is useful for separating certain hard-to-tell conifers from each other. This is applicable also, with small-dimension hardwood specimens, as the branching pattern is a constant within tree families but varies between them.
Wood density varies immensely between species, but also within species. Roughly estimated density can still be useful for further ID. With generally relatively featureless diffuse-porous woods, where finishing can easily hide subtle features, the density difference between, say, alder and birch is evident.
ID’ing wood to within genus with these and other “field methods” is doable, but there are hundreds of species of oak and pine, for instance, out there. For positive, accurate ID, microscopic analysis is needed. Even with it, certain closely related species can’t be told apart.