Unfortunately, that isn't good enough. Any government has a library of exploits in free software, and there are many documented cases of this. Your encrypted files are only as good as the clients that access them, and that's where the weaknesses lie. Along with the traditional aspects of surveillence, of course. "Amateurs hack systems; professionals hack people."

For the vast majority of people, the hassle simply isn't worth it, especially when the massive reduction in convenience is paired with basically no improvement in security.