I admit I don't have a ton advice regarding saving money while traveling, since most travel, even the most basic, is generally out of our financial reach. (Also, traveling with a toddler? Not easy. It can be fun, for sure! But not easy.)
My one big tip for locals (and others?) is to check megabus.com for bus tickets. For my recent jaunt down to Boston (to catch a train to Connecticut) I paid a grand total of $13.50 for my round trip bus tickets when I would normally have spent $36! Same bus service, same everything – just a cheaper price! I must admit I was wary at first…would they treat me like a second-class citizen when I showed up with my megabus confirmation? Would they even let me on the bus? But I needn't have worried. It was just as easy and seamless as having purchased my tickets at the station, from the carrier.
The other recommendation I have is that if you have a baby, travel with them before they cost any money! Generally speaking, this means before they turn two. I took E on a train trip when she was 5 months old and it was great fun. She was still little enough to be highly portable but old enough to sit on her own and not fuss too much. (This will, of course, depend upon your particular baby's mobility and temperament…every baby is different!)
Good luck and have fun!