I happened to install ubuntu 64 bits at work some months ago. Installing flash in a 64 bits linux distro is not straightforward and I haven't dedicated the time yet - after all, I seldom need to browse a flash site. But every now and then I happen to browse a website with a big white hole in the middle of the screen and a big NOTHING filling it, and have to guess what I am missing. It's okay (and expected) if I cannot see the last YouTube thing, but requiring flash to see a city map seems like a misfortunate decision. I ended up annotating the address and going to Google Maps instead.
Think about accessibility like this: a little amount of possible customers that are going to move to other sites because your site is thought, designed and oriented for WASPs. Cell phone browsers do not have a mouse, the elderly may prefer bigger fonts, some users cannot have flash installed because his sysadmin is a security freak concerned about demons and aliens invading this dimension (not my case, again; I'm lazy).
Your environment is not THE environment. There is a consistent percentage of your market that is going elsewhere - not because they don't like your website, just because you don't care.