I can't seem to find the link now, but there was a post here a couple of weeks back about someone doing a very similar thing (in Python, IIRC) to clean up pictures of handwritten notes. It seemed conceptually simple, and worked quite nicely.
Wonder how close this is to the techniques apps like Office Lens use. I am always amazed to the extreme angles I can take a photo and the app still makes it end up as perfect pdf.