Professional compiler engineer here, C is a mediocre intermediate language.
Let's start with an excellent quote from Wittgenstein.
"The limits of my language mean the limits of my world."
Using C as your intermediate language means that your expressiveness is limited to valid C programs. This is workable but only if your language can be mapped to C in _useful_ ways.
For example, let's say your language has behavior similar to scheme's tail-call.
How would you get this behavior from a C compiler? You will never be able to make this reliably across optimization levels, etc.
Guaranteed tail-calls are the tip of the iceberg, there are a lot more features which cannot be reasonably mapped onto C.
Real compiler IRs increase your expressivity beyond what the C language designers decided was important.
Exactly, and this is what most people who work on compilers discovered long ago. It's disappointing to see "just compile to C!" still so popular on HN.
I would cite GC as the most prominent example of a feature that is incompatible with compiling to C. It is impossible to write a competitive high performance tracing GC on top of C. Mandatory register spills at safe points and conservative scanning have huge downsides.
> It's disappointing to see "just compile to C!" still so popular on HN.
Posts on C get blindly upvoted on HN and proggit. I guarantee it. Me thinks it's like the history channel for programmers; learning about their forerunners and how they had to bang rocks together to make fire.
It's also how I've gotten all of my karma. Unfortunately, the abyss has also looked into me, as I now write C code for a living...
Using C as an intermediary language can still be useful at times. For example, when writing a bootstrap compiler, you can target C and magically support virtually every platform. It's not so great for production compilers for numerous reasons, including performance (poor unless your languages maps cleanly to C) and debugging experience (debugging object code is not fun, and if your object is C, you get stuck with whatever debugging information your C compiler outputs; you don't have any say in the matter). For bootstrapping compilers, I only see three sane options, really:
1. Write an interpreter to host the compiler (probably in C).
2. Write a cross-compiler.
3. Write an X-to-C compiler.
I insist on C here not because of performance, but because it's ubiquitous.
> For example, let's say your language has behavior similar to scheme's tail-call. How would you get this behavior from a C compiler?
Well, there's still the unreasonable way: compile in continuation passing style, and trampoline every call. I shall not be held responsible for any performance problem this advice may cause however…
As far as I can tell, on all the axes that LLVM excels as an intermediate language (ease of getting started, debugging support, many backends, optimization, flexibility), C is even better. C is easier to get started with, has easier debugging support, has more backends, better optimization, and more flexibility.
To take a simple example, I have here a 2428-line C program generated by compiling Linus Åkesson's Game of Life in BF (http://www.linusakesson.net/programming/brainfuck/) into C using Daniel B. Cristofani's dbf2c.b (http://www.hevanet.com/cristofd/brainfuck/dbf2c.b), which is a BF compiler written in BF. Compiling these 2428 lines of C to machine code using tcc 0.9.25 takes 20ms on my 1.6GHz Atom netbook. Most of this is about 16ms of tcc overhead (startup and shutdown time); the rest is compiling several hundred thousand lines of C per second with tcc. You should get several million lines of C per second with tcc on a modern machine.
This isn't optimized code, about equivalent to gcc -O0, typically about 3×–5× slower than optimized code. But that's enormously better than interpretation overhead.
(Using decent optimization levels with GCC makes it take several seconds to compile, because GCC's optimizer doesn't deal well with enormous functions.)
dbf2c.b, the C-generating BF compiler, is 892 bytes of BF code when stripped. Now, I'm not saying you should write your compilers in deliberately obfuscated programming languages in as few bytes as possible; I'm saying that the fact that this is even possible at all should give you really good feelings about how easy it is to compile things to C.
What? How does C have "better optimization" than LLVM? As far as I can tell, this seems clearly false: you can't do custom alias sets, for instance, in C, whereas you can in LLVM. None of your examples mention optimization at all (and BF is also totally unrepresentative of anything in the real world).
Also, C may have easier debugging support, but it's significantly worse than having exact control over the contents of your DWARF DIEs. Making it easy to have a bad debugging experience isn't a plus as I see it.
C has better optimization than LLVM because if you generate LLVM IR, you can't compile it with icc or Visual Studio. (Or GCC, but LLVM probably already equals or beats GCC at optimization already.)
You're probably right that LLVM also has better optimization than C because of things like the possibility of doing better aliasing.
I agree that BF is unrepresentative of anything in the real world. Thank goodness.
I can certainly see how debugging support could be better if you can go beyond the support C gives you.
Indeed. C is brilliant as an intermediate language. One of the greatest things about it is that once you generate C code it's not too much of a stretch to also generate C++/Objective C specific code and take advantage of the libraries written in those languages.
Despite this, there are many people out there who view languages that compile to C as being inferior. I still don't understand it.
There are many things that are present in assembly but lacking in C. For example, how do you detect integer overflow in C without resorting to compiler extensions?
I would like to see the benchmarks before I agreed to that.
You might not get the best optimization possible if you have to leave it up to the C compiler with inline assember... But that mix worked well for C and C++.
And rust can still optimize things like deciding which integer optimizations need the overflow check, and only add it where static analysis failed.
No, it doesn't work well for C and C++ at all. Turn off InstCombine and see what performance you get in LLVM for an idea of what will happen if the compiler doesn't know about the semantics of + and -.
We are only talking about a subset of additions and subtractions to begin with.
How often will idiomatic rust code need overflow checks?
Also, if the UB operations are defined by target assembler (instead of in C++) then you may not even need extra overflow checks, depending on the assembler semantics. This would only be needed in the places where static analysis failed, of course.
One could also use the native compiler options, if available, to avoid the issue alltogether (like -fwrapv).
> We are only talking about a subset of additions and subtractions to begin with.
No, you're talking about all signed addition.
> How often will idiomatic rust code need overflow checks?
You'd be surprised. Go look at the implementations of containers in the standard library.
> Also, if the UB operations are defined by target assembler (instead of in C++) then you may not even need extra overflow checks, depending on the assembler semantics. This would only be needed in the places where static analysis failed, of course.
I don't know what this means exactly. If you're saying you should implement a static analysis to try to eliminate unneeded overflow checks, then this is a huge burden.
> One could also use the native compiler options, if available, to avoid the issue alltogether (like -fwrapv).
If you're dependent on GCC/clang extensions to C, then you're not compiling to C. You're compiling to a front end to GIMPLE/LLVM. At that point there's no benefit over just compiling to GIMPLE or LLVM in the first place.
No, I'm talking about the ones that won't overflow. Some can be statically shown to not need a runtime check.
> If you're dependent on GCC/clang extensions to C, then you're not compiling to C.
I'm not saying that. I'm saying require either the target compiler (which the user is in control of) to provide a flag to give signed overflow the proper semantics (which would be useful if the user happens to be using clang or gcc), or fall back to requiring some asm for the target system which provides the needed semantics.
This would all be up to someone to configure for the target machine. Not a big deal once per architecture.
> At that point there's no benefit over just compiling to GIMPLE or LLVM in the first place.
Well if GIMPLE or LLVM was available for all of the targets that C compilers are, then I'd agree.
could be implemented in plain C if you're careful. It's not like the other extensions like __builtin_constant_p or asm that'd leave you stuck with GNU C.
And besides, GNU C is good enough for lots of people!
A lot of the claimed benefits of compiling to C--compilation to embedded targets without a GCC backend, most importantly--evaporate if you only compile to GCC's dialect of C. It's another way in which compiling to C ends up worse in the long run.
C might be decent for the short term, but if your language is at the stage of trying to be adopted, I'd rather see some IR output with a C ffi. You have to take into account what semantics you are saddled with as you go through the translation process through C, for example all that damn undefined behavior...
It's also insanely easy to generate code for. In addition to this, you don't have to build an entire toolchain of LLVM stuff to get it to work on your system -- LLVM is a nightmare to setup on Windows.
Portability and interoperability are key in industry.
I think all of the new languages are amazing these days. I am learning Rust, Clojure, Haskell, and many more.
Buy I can not use any of them at work. I'm in a position to influence teams of smart programmers, but the only language that I would be able to use would be one that fits in to the existing infrastructure, which is natively compiled shared libraries on esoteric unix platforms.
So no LLVM. No JVM. No Haskell. I could possibly get away with a lisp or scheme that compiled portable C if I wanted. But that doesn't excite me as much.
I would kill for a Rust to human-readable C++ transpiler. I think it could be used immediately by many.
Note that Rust code can interoperate with other languages by exposing a C interface and having the host language call it as though it were C. There's no runtime to start up or anything, so to the host platform there's no discernible difference (though you may need some machinery in your build system). This is how Rust-written libraries have been integrated into Firefox (which is obviously still integrated into Firefox), and there are companies that have been using Rust to extend Ruby for years now.
Yep. I understand and this is why I could integrate rust right now into my products, module by module or object file by object file.
I just need the compiler to run on my systems. Since it is written in rust, I either need it ported to my systems (too much work) or I need my rust code transpiled to C or C++ so I can build it myself (seems doable).
Do you actually need a compiler that runs on those exotic systems, or just one that targets them? Because Rust (thanks to LLVM) is a whole lot better at the latter than the former, and it might well be possible to build something for your obscure UNIX. But for various reasons, Rust is not great at all at cross-compiling itself, so you would likely have to use something more conventional as your build server.
It's not doable. How do you deal with UB from signed overflow? How do you get proper debug info? How do you set the proper EH personality?
There are no shortcuts to a proper implementation of Rust. Implement an LLVM backend for your architecture. It doesn't make sense for Rust to maintain a backend that is doomed to be forever broken in various ways.
I answered your ub question elsewhere. Exceptions and debug info come from C++. I don't see the issues there yet, but perhaps I just don't have the right knowledge yet.
Rust won't have to maintain the backend. Who said they would? They might choose to if it becomes useful - it would give them things they dint have now (C++ interoperability, and new platform support for almost free), but no one here asked them to maintain anything.
I am also not sure what you mean by shortcuts or broken forever in various ways.
> Rust won't have to maintain the backend. Who said they would? They might choose to if it becomes useful - it would give them things they dint have now (C++ interoperability, and new platform support for almost free), but no one here asked them to maintain anything.
If out of tree, the backend will constantly break due to massive compiler churn. You need it in tree.
I don't think so. The backend would simply lag behind major releases a bit. It's not going to use all of the compiler internals, so updating it will only be a little extra work if the HIR (or maybe MIR, depending) changes.
If Rust has plans on building a large amount of their internal optimizations on MIR, then I would think MIR would be fairly stable too, which makes this even less of an issue.
It's going to be a toy project for now. If it works out, it'll be maintained against a specific version of the Rust compiler (stability is important in industry). And if that works out even more, perhaps Rust will accept patches to put it in-tree.
Of course, in-tree would be better. But it isn't a requirement.
Rust is interoperable. That's why I could use it. I just need the compiler to generate code I can build with my C or C++ toolchain, since there is no rust toolchain for my platforms.
The problem is that usually not very human readable C comes out of it; to use something instead of your company's language of choice, unless you are the CTO or something else high up the ladder, would need to compile to the company's language and do it in a human readable way. Some Javascript 'transpilers' do that (I think Coffeescript?) but usually it really acts as a compiler and the idea is that no-one should need to read / use the source code that is emitted besides for running it.
> The problem is that usually not very human readable C comes out of it
The x86 assembly or LLVM intermediate representation that comes out of most compiler is not very human readable, but I don't see anyone having any problem with it whatsoever. Readability of generated code was never the point of C using C as a back-end. The point is portability and tooling.
By the way /u/jjnoakes can suggest fancy languages and compilers, if only they can generate native code on obscure platforms (I guess, something other than x86 and ARM). Generating C code (even utterly unreadable C code) would solve his problem.
Remember, it isn't only the architecture. It is also the calling conventions and object/shared library/executable file format.
All things that require work if I cross compile, but all things I get for free if I generate C or C++.
I would prefer readable C or C++ as much as possible because it gives me debugability if the code generator goes wrong and it gives me an out if I want to move away from the language some day.
Well, there aren't many solution out of this hole. If you need another language, you might have to write (and control) the compiler yourself. As in, if the code generator (or anything else) goes wrong, you can correct it.
<sarcasm> Now this might require a little bit more work. </sarcasm>
>I'm in a position to influence teams of smart programmers, but the only language that I would be able to use would be one that fits in to the existing infrastructure, which is natively compiled shared libraries on esoteric unix platforms.
So, how does that prevent you from using natively compiled languages like haskell?
What systems are those? And why do you need "human readable C"? You said you needed it to work in a natively compiled shared library environment. You do not need to compile to C for that. But if you really want to do that for some reason, you can do it.
I'd prefer it to be human readable because it helps in debugability if the code generator is broken (or we misunderstand some subtle semantics and want to look closer), and if I ever want to move away from the language in the future.
Ignoring that, if I don't compile to C or C++, then I'm not sure how easy it will be to use. Are you suggesting Haskell can generate object files, shared objects, and binaries for my systems without compiling to C or C++ first? How would it get the file formats and calling conventions right? Can I specify them somehow to Haskell (doubtful)?
ghc by default can generate object files for systems it doesn't know about? That sounds positively magical. How does it work?
> The same way every other compiler does.
Every other compiler that targets native code for these systems has first-hand knowledge of the file formats. If Haskell doesn't require that knowledge, I would love to hear about how it works.
Here's an example. Say Haskell only supported Windows on x86_64 and I'd like to build my Haskell program for Linux on PowerPC. Are you telling me Haskell on Windows could generate Linux on PowerPC executables and shared libraries without first-hand knowledge of the instruction set, calling conventions, and the ELF file format?
I don't know why you are being so hostile and evasive at the same time. What systems are you talking about. I will tell you if there is a compiler that supports them. That's the problem right? That it will turn out that your obscure systems are very normal and supported. Silly fallacious replies about magic are not constructive. Just answer the question and I can help you.
I'm not being hostile at all. If such a thing exists as I believe you described it, I would find it to be magical and would love to use it. I would even pay for it.
And I'm not being particularly evasive either. I gave you an example which fits my situation perfectly. I'd prefer not to reveal the names of the actual systems I need to build for, since it would fairly uniquely identify me.
Are you willing to work with the hypothetical I put forward? Or are you dodging that in favor of attacks on me personally?
Every native compiler does this. There is nothing magical about it. Obviously the platform has to be supported. That's why i asked what the platform is. The idea that you are the only person running a system is absurd. Even the most obscure platforms have thousands of active users. Your hypothetical can not be worked with, because it is a concrete question. Is your platform supported or not. Very simple. And again, if you want to have it generate C, then do so.
You think I haven't tried using Haskell on my platform? It isn't supported and it doesn't work. So you misrepresented things.
I mean why would my first post be "Boy I wish I could use Haskell on my platform" if... I could use Haskell on my platform?
Now I could put a ton of effort into it... for each platform... but I don't have that kind of time to maintain Haskell for N platforms.
It would be much easier to compile via portable C or C++ that was human readable. But I would expect, since Haskell has lazy evaluation and a GC, that there is almost no way to map to readable C or C++. Thunks flying around, continuations, ...
>You think I haven't tried using Haskell on my platform?
Correct.
>It isn't supported and it doesn't work
What is "it"?
>I mean why would my first post be "Boy I wish I could use Haskell on my platform" if... I could use Haskell on my platform?
Because you incorrectly believe you can't.
>It would be much easier to compile via portable C or C++ that was human readable
Then do that, like I told you repeatedly. The fact that you have one of the developers of a haskell compiler trying to help you and you are entirely hostile and evasive makes it hard to take you seriously.
Mentioning is not helpful, telling is. So tell. What format? What instruction set? What compiler?
>You keep claiming Haskell solves my problems
I claimed nothing. I asked you a simple question to try to help you. And look at your responses. You still haven't answered a single basic question.
>And I'm hostile and evasive.
Yes. You refuse to answer even the most basic question that is needed to help you. You do not want help, you want to make a vague, baseless complaint and then avoid having that complaint proven baseless by refusing to provide any information.
I've provided enough information to get to the right answer (Haskell isn't magical and can't compile to my platforms if it never heard of them, and it doesn't generate portable and human readable C or C++ so I can't build Haskell code using my tool chain).
That's all that matters to me. I'm not sure what crusade you are on... but if you had read the comments you replied to you would have avoided wasting my time and yours offering up Haskell as a solution when, based on information contained in the very posts you replied to, you should have been able to figure out that such a position was a misrepresentation.
Whether it was intentional or not I will refrain from speculating.
Your characterization of the discussion is not accurate. Read the thread, it is still here. You lamented not being able to use haskell. I took this in good faith and tried to help you to use haskell. You have done nothing but deflect and derail trying to avoid being helped. You insist haskell can't do this, but you still can't even tell me what haskell compiler you even tried. There's dozens of supported platforms, and more than one compiler that produces C. You are being dishonest because you never actually wanted to use haskell, you just want to post vague anti-haskell nonsense.
>Haskell isn't magical and can't compile to my platforms if it never heard of them, and it doesn't generate portable and human readable C
Haskell isn't a compiler. Different compilers support different platforms. You insist "haskell" has never heard of your platform, but I guarentee you can't find a platform that I can't compile haskell code on successfully. You are simply being dishonest. And yet again, yes, multiple haskell compilers produce human readable C code as output.
No, you made claims like Haskell natively supported compiling to the targets I am using without even knowing what those targets are. That's not helpful to anyone.
> You have done nothing but deflect and derail
Also not true. I provided plenty of information and offered up an analogous situation. You decided to ignore that, and ignore the other things I said, and you continue to attack me instead.
> multiple haskell compilers produce human readable C code as output
If this was true all you had to do (8 posts up) was mention this. Why all the cloak and dagger? Why all of the ad hominem attacks and misrepresentation? Why not just offer up a simple solution that you know about?
I'd appreciate a list of which Haskell compilers produce portable human readable C code. I'd love to use them, if they exist.
Next time someone says "I'd love if Haskell could compile to human readable C or C++" and you know of an implementation of such, it'd be helpful to mention that, instead of tilting off on whatever personal crusade you imagined up against me.
That's right, I refused to, over and over. The answers are irrelevant.
> You have provided one piece of information
That's right, and zero pieces of information were required for you to be helpful. You claimed you were trying to be, but your actions speak much louder than your words.
> Neither is lying.
You claim repeatedly that Haskell compiles to readable C. It doesn't, which is why you can't list any single implementation which does.
That does not say anything of the sort. It says there are haskell compilers that compile to native binaries and interoperate in a native code environment. You simply refuse to specify your platform so you can try to maintain the illusion that someone would still think you aren't full of it. You aren't even fooling yourself.
>That's right, I refused to, over and over. The answers are irrelevant.
Not if you actually wanted help they aren't. But since you actually just want to shitpost, all your refusal does is make it clear you are dishonest.
Me: "Are you suggesting Haskell can generate object files, shared objects, and binaries for my systems without compiling to C or C++ first?"
You: "Of course, that's what ghc does by default."
And now you claim you were not saying "anything of the sort" when I point out that you claimed "Haskell natively supported compiling to the targets I am using without even knowing what those targets are"?
> Not if you actually wanted help they aren't.
100% they are. Unless you can explain how me naming the systems I support has any relationship to whether or not some Haskell implementation has the ability to generate human-readable C code?
> But since you actually just want to shitpost
The only shitpost here is you backtracking your claims repeatedly.
Unless you listed those Haskell implementations you claimed existed somewhere and I missed it?
But on the platforms it compiles to, Ocaml can generate object files that look like they came from C. I use this feature right now to integrate a compiler in an otherwise C++ project. Here's a link to the FFI documentation: http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html
But I run on Debian stable on x86-64. Not exactly an exotic platform.
This begs the question of what would be better - to compile your favorite language to C and use something like Emscripten to make JS, or to compile your code directly to JS?
So far as my experiments went, a 3000 native score Dhrystone test compiles to C at 1300, and that C compiles to 850 worth of JS. Which suggests that a direct-to-JS compiler might be worth the efforts...
really wish this was more common, as language choices on embedded platforms are often VERY limited. ie, C.
Though on embedded systems you'd also want more constraints for the backend, like do not use dynamic memory, perhaps being able to specify the output code is MISRA compliant.
Let's start with an excellent quote from Wittgenstein. "The limits of my language mean the limits of my world."
Using C as your intermediate language means that your expressiveness is limited to valid C programs. This is workable but only if your language can be mapped to C in _useful_ ways.
For example, let's say your language has behavior similar to scheme's tail-call. How would you get this behavior from a C compiler? You will never be able to make this reliably across optimization levels, etc.
Guaranteed tail-calls are the tip of the iceberg, there are a lot more features which cannot be reasonably mapped onto C.
Real compiler IRs increase your expressivity beyond what the C language designers decided was important.