That sounds like someone from the suburbs. 
I’ll give a long answer, but the short answer: no, it’s not crime-ridden. Yes, there’s a homeless problem. But even in the best of times, downtown isn’t really a tourist destination with the exception of the waterfront (Tom McCall Waterfront Park) and Powell’s Books, which is getting toward the edges of what I’d call downtown. The other places mentioned above like the Rose Garden and restaurants aren’t in downtown. I don’t avoid downtown, but I don’t really have reason to go.
For the longer answer: downtown has a few issues right now. First, there are a few remaining effects of the protests, in graffiti and boarded up windows. But this is a localized area of just a few blocks. In most of downtown, you’d never know anything happened.
Second, Portland took the pandemic very seriously and pretty much entirely shut down. There are still very few office people working down there, which means stores and restaurants that rely on foot traffic are also closed. Many of them boarded up their windows during the long closure. So downtown has a bit of a feel of a ghost town, and any garbage and graffiti becomes more noticeable.
The biggest challenge is the homeless problem. Portland, like most larger cities, has one, and it’s gotten worse in the last year. This has been exacerbated by the city’s policy to put a moratorium on evictions and cleanups of homeless encampments during the pandemic. So there are a lot of tents in some areas, and some of the encampments can fairly be described as overrun with garbage.
The city announced this week that it is resuming evictions and cleanups, so it might be better when you come. But without major changes, even the evictions just result in pushing people from one place to another.
In general, the homeless population doesn’t represent a physical threat. The few times that someone has a mental health crisis and accosts a passerby are rare enough to be news. Certain media outlets like to play up those incidents, but they are very rare.
But I will agree, it is not enjoyable to walk by so many homeless people. It’s a constant reminder of our failures to provide an adequate safety net for economic and mental health issues, and it’s guilt-inducing. You will see homeless people outside of downtown, so it’s worth it to prepare your boys for the experience if you don’t live in a place with many. I think it’s an opportunity to teach compassion over fear, but the topic is unfortunately politicized, so I’ll leave it at that.