Yeah...just keep at it you'll get it out. This is just an observation on my part but trees are easier to remove with root ball intact when they are standing and not sawed off. It takes a pretty good sized machine with a lot of break out force to pop a stump that sized out of the ground. You'll just have to work on one side and get as far under the stump as you can then work your way around and under the root ball until you get enough dirt removed where the stump itself is light enough for you little machine to be able to handle.
When I started my barn the site was full of huge oak and pine trees. I didn't have enough money to hire someone with a track loader or excavator to clear my lot so If it was to be cleared it had to be pretty much cleared by hand. It took me from late Aug. through all of the winter and into March of the next year to get all those trees out of the ground and the stumps burned. Some of them were so big all I could do was pull them over and burn the stumps right where they fell several times to finally get them light enough to be able to move them. Then before I could ever burn them I had to stand there for a day, sometimes longer with a pick and shovel removing as much of the dirt as I could. Dirt doesn't burn too well and on top of that I needed all that dirt to fill the massive hole that was left in the ground.
I got all those trees out with a 60' long piece of 1/2" cable, a couple of 20' log chains, a few heavy nylon straps, and my trusty 044 stihl chainsaw. Oh and I forgot, my old 1996 Bronco that I used to pull these big trees over with. Pulling them over wasn't too bad it was the moving them around once they were over that was hard on the poor ol' girl. But I got it done and so will you. You just have to stay with it
Oh.....and I like the idea of the crushed concrete for fill material. That stuff works really well and in most cases it's a lot cheaper than crushed stone. Just make sure that if you are going to have plumbing or conduit under the floor get that in before you raise the grade up to sub grade level. it will be a lot easier to trench the piping and conduit in the dirt than it will be through that stuff once it sets up. I would build the whole pad with it to within a couple inches of sub grade then use sand, stone dust, or light crushed stone to do your fine grading with. It's much easier to do your finish grading with than that broken concrete is. Oh....and you may already may know this but just in case be aware of the pieces of rebar and chopped up pieces of reinforcing wire that will come with the crushed concrete it's h@ll on rubber tires. It's pretty easy to see and pick to up just keep an eye out for it.