Someone commented on collecting wild bird eggs. This is likely to be highly illegal as most birds are protected. Unless you live in a heavy agricultural area, pesticides aren't much of an issue. I wouldn't want to eat city pigeons though, though with baiting, they would be easy to snare.
There used to be a lot of hunting, but costs and crowding have cut way back. Unless you can hunt on national forest land in your own state(USA at least) licenses and leases can make it a rich man's game.
In the south bag limits on deer in some counties are amazing - nearly unlimited. Wild hogs are unprotected, but very wary. Unless you are a very good hunter with good locations, it can be pretty hit and miss - mostly miss. Unless you are very experienced it can be very time consuming to scout out locations and actually find something. In some areas(Texas) feeding is allowed, which guarantees success. However, nearly all hunting is on leased land. Its not unheard of to sneak a salt block to encourage showing up.
A group effort can be useful. join a drive.
In squirrel used to be common fare in the south. They could never be hunted to scarcity. Ugh! - Rats with tails!
If you put out bird feed and don't make it too squirrel proof you can get almost unlimited squirrels. Just don't let them see others in the traps.
If costs are your concern, I would go Neolithic and raise your own. Rabbits are quiet and can be raise in the city. Chickens are a bit noisy. Doves are pretty quiet. You could raise quail and feed them on earthworms and crickets to get away from a totally grain fed meat. I would look into aquaponics as a way to raise both fish and Veggies.
I don't think I will go this way though I could. I have deer moving through my backyard nearly every morning. Also rabbits and plenty of turtles, alligators,and fish in my pond and the nearby canal. I'm too much of a Vegetarian!
I really don't enjoy ending another life even to eat, though I have hunted and fished at times.
John