I was thinking something similar to Larry. Having three feet of exposed beam from the ground the the skirt has the potential to cause problems. PT wood likes to curl on it's own, and this might give it enough room to do so.
If I absolutely had to start construction before the pad was level, than I'd put larger poles in the ground. At least one size bigger than you need. I think you'll need the extra strength.
It sure seems like a much harder way to go about building a barn. Moving the dirt in later and compacting it is gonna be ten times the work and effort having to work around the walls. Compaction will be just about impossible when your trying to get it from both sides of the walls, which means it will continue to settle for years, even decades. No matter what, you'd never be able to pour a solid foundation tha wouldn't crack on you over time.
Anything's possible, but why do it the hard way?
Eddie