You need a meter or at least a test light... why go probing around like a blind man in the dark, looking for something you can't see, hoping you stumble across something that might work? Meters are cheap, and you aren't going to get away from electricity issues.
Troubleshooting 101
1. Verify you meter is good, by testing on a known good 12v source, like your car that starts
2. Check across the suspect batteries on the terminals themselves, then the clamps.
3. Verify a good ground by keeping + on positive post, and move - to somewhere on the engine, which will verify that you have a good ground link. Then you are confident in using any ground point on the engine for your - probe.
4. Start from the battery + post, working back each section to find out where you lose +12v. Normally, there should be a direct wire from the battery + to the starter or starter solenoid.
And yes, you can have a bad new battery. Lawn and Garden batteries are the absolute low end of the battery world. I replaced all my L&G lead acid batteries with ATV/motorcycle AGM batteries, which last a lot longer, are much more vibration resistant, and are usually smaller dimensions, and sometimes cheaper. Although, you probably have an automotive battery if this is a tractor and not a mower, so you should be able to swap any automotive battery with it that will fit in the spot and fit the terminals, or like some have suggested, give it a jump. And keep a battery maintainer on them.