The weird thing is Kiyohiko's past work was basically only Azumanga. Looking at his Wiki page there was only one other manga (Wallaby) which only had one volume as far as I can tell. It's insane he's had a 20+ year career and only has three series because he's been working on Yotsuba for 80% of it.
I dunno if that's comprehensible, so I'll say it again: thanks for the advice! I feel better knowing it took 2-3 years to get to some sort of fluency. I'm like 1.75 years in, although I've only been studying daily for just under a year. Anki is a life saver. I've been doing a vocab deck with kanji since near the beginning which was the best decision I ever made. I also started watching anime with Japanese subs recently, although if it gets very dialogue heavy it's slow going.
I kinda wish I'd started with the Genki textbooks, but now I'm as far in as I am I don't want to shell out $60 for something I probably know half of.