Oh! Oh! I have a story!
So I’m in college, and of Asian descent, but born in New Jersey. My parents thought it was in my best interest to NOT teach me any Chinese while growing up to prevent confusion between languages. The University I attend requires a certain amount of foreign language to be taken in order to graduate, unfortunately I realized that at the end of my junior year.
So I tried to pull the race card, and went into the Chinese department, and I told them, “Yes, English is my secondary language.”
Now everything is going until the department decided to call me on my bluff. So who did they call? The department Chairman.
After failing miserably, he told me to come back in about three months and try the written test. As soon as I was out of his office, a angry call was made to the parents and in the span of three months, I learned to level two Chinese.
It made almost no sense, Chinese, I mean. There are radicals, little modifiers to be added onto the different characters, different strokes to remember. Thinking about it now is giving me flashbacks. Ugh
A few (4-5) people I’ve known have told me “English is very hard to learn”. Not necessarily harder than any other language, but very hard. If you think about it, there’s a ton of words that have multiple or odd pronounciations, regional expressions, slang, and other odds and ends that don’t follow our own language rules. There’s tons of filler phrases like in, of, to, at, etc that are used multiple different ways. Loosen and unloosen mean the same thing, but tie and untie do not. English, especially the Americanized <tm> version of it, is a strange language with tons of quirks and oddities that take a long time to master.
All languages have quirks and oddities. I don’t think English has more of them than most languages.
I know of at least one university where a student who tries that will get expelled, so hopefully nobody else will do that either.
Glad you did get some benefit out of it, though. There is sense to the radicals in written Chinese: they’re used to classify the character in dictionaries. Personally, I prefer the SKIP system for that, but “different strokes for different folks.”
Of course, one does tend to feel that one’s own language has special qualities. I too suspect that English is a doozy for those who didn’t grow up on it, because when I come upon a Latin idiom whose meaning cannot be gleaned from merely understanding the vocabulary or grammar, I can think of a dozen similarly baffling English expressions. This is a natural consequence of the process: Since studying a foreign language necessarily focuses on the regularities to set a foundation on which you can tackle the exceptions, you just won’t be confronted with the weirdness often, yet as a native speaker you will have a vast knowledge of quirky usages to call upon there. So, your own language from such a perspective will almost certainly seem more confusing under that kind of focus.
As an example, I came across an expression “actumst” in Latin. Well, that doesn’t look like Latin, it looks like German. Turns out, it’s an unmarked elision of “actum est”, which literally means something like “the thing was done”. But it further turns out that when someone says this, they mean that somebody is doomed. You hear “Actum est de <somebody in the ablative>” and you know somebody is done-for. Oddly enough, this same expression was used in an Asterix comic where the Romans were playing a game of musical chairs to select a candidate for a mission, and in this context the phrase might have stood as the Latin for “You’re it!”
In English, of course, “do” means many things that are not systematically encoded – the only accounting for them is to memorize a bunch of idioms and contexts. Sometimes “do me” means “zip up my dress” yet it can also be an exhortation to do quite the opposite. A native speaker would know without having to think about it. You have to get pretty deep into a foreign language before you really get immersed in these kinds of subtleties, so it just seems like there are more of them in your own.
Here’s a related issue: I have found that German speakers tend to be convinced that their language is just untranslatable. I mean, everyone’s language is in a non-trivial way, but Germans seem particularly keen on this point. Take “zietgeist”, which in English seems to mean what it sounds like - ‘the spirit of the time’. Well, apparently not. Germans insist that the word has so many deep hooks in the German language and culture that you can’t possibly make sense out of it without understanding German. But frankly I suspect this is really just a kind of shared attitude than a special quality of the German language.
I think English actually is better than most languages in that regard. Germans have to make do with a word that could mean “spirit of the time” but to them means something more inscrutable. In English, we can just say “zeitgeist” wholesale without needing to worry about such double meanings 
I think most native speakers of languages believe their own languages are particularly difficult to master; I think it’s a desire for some sort of exclusivity, implying superiority.
The catch is that verb “master”. Learning to speak a language (i.e. to form grammatical sentences in a variety of contexts and on a variety of subjects and be tolerably well understood) is generally not too difficult, but mastery is both difficult to define and difficult to achieve.
For the sake of discussion , let me suggest that a person can be said to have mastered a language with the ability to understand and employ nuance, understand and use a wide range of idiomatic expressions, perform fine social manoeuvres, and produce intentional original humour. This involves, of course, not only a fairly large amount of technical linguistic knowledge but also a fairly large amount of assimilation of cultural knowledge; language, in the end, is not entirely separable from culture. On this basis, I would say that all languages are difficult to master. The difficulty comes not from the language itself (though certain aspects of some languages, such as Dine’s mindbending verbs, Russian case inflections, or Cantonese tonal structure, may present difficulties to those who find them alien), but from the lack of cultural grasp.