So far so good but would never ever recommend aluminum conductors. There will be a need to care for those connections to keep them tight., Anti-corrosion coating.
It would may be better in the long run to install a 100 amp panel. It should about 8 circuits for you.
Run the #4 awg copper cable (within conduit or not) and be set for a 120/220 source at the panel and be done.
Because, one's original plan are only warrantied until the moment it is completed. Immediately after that one moment in time, murphy's law says:
Hmm, air compressor would be handy here, hey a welder would come in handy, see the path here. Which would indicate one size large for a safe margin of safety if cost is not prohibitive for you.
So be safe, use correct gauge and insulation rated conductor for underground install.
Ground at barn be set for 120/220 vac and you set for unforeseen need and growth at least a few years.