Notorious has at least two similar meanings; the more common one is "known unfavourably", the second meaning is a neutral "known", not positive or negative. Most people only see the word used in the negative light so shy from it in the neutral use case to avoid being ambiguous or assumed negativity.

Using it with an adjective to clarify the meaning here is fine.

As a non-native speaker, my understanding of "notorious" so far has been what you described as the neutral form. However, to me, it does have a slight connotation of obsession or "doing it so much as to be annoying".

I am obviously not an authoritative source on how words are understood, but just wanted to add this data point.