Modern banking systems allow the banks themselves to 'print' dollars - by originating loans. The Fed (and central banks elsewhere) steer this money creation process through various mechanisms. However it is NOT the case that 'every dollar must be paid back to the Fed'.
The reason for this expanding money supply is to facilitate an expanding economy.
You are right, part of every dollar must be paid back. If there is any interest on a loan then you need to pay back more than 100% the original cost.
Our FED operates the same way, the government must pay back the FED's loans, and these loans have interest. When the US was printing greenbacks without the FED there was no one to "pay them back to". This is how the USA operated before 1913.
Inflation existed before 1913. What was different was that inflationary periods were counterbalanced by periods of deflation.
There's a reason why the Federal Reserve has two primary missions--stable prices and maximum employment. Deflationary periods cause massive unemployment and declines in investment, so they really want to avoid deflationary periods. Ask any economically literate historian about the devastation deflation causes at the individual level; it's far, far worse than inflation at nominally comparable levels. This is why the Fed is biased toward inflation. 20th (post-Bretton Woods) and 21st century inflationary periods haven't been any worse than the 19th century or early, but we have avoided severe deflationary periods, and that's absolutely worth the small long-term inflation. It's also arguably kept prices more stable, but confounders (technology, etc) make comparisons with the 19th century industrial revolution complicated--the emergence and rapid evolution (to this day) of the commodification of goods and services.
Try reading, Lords of Finance: The Bankers Who Broke the World. It won the 2010 Pulitzer Prize for History, among other accolades. I certainly enjoyed it, though I read it on the heels of the 2008 financial crisis, which may have made it feel especially salient.
The reason for this expanding money supply is to facilitate an expanding economy.