Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Whilst the C standard doesn't guarantee it, both LLVM and GCC _do_. They have implementation-defined that it will work, so are not free to optimise it away.

[0] https://llvm.org/docs/LangRef.html#llvm-memset-intrinsics

[1] https://gitweb.git.savannah.gnu.org/gitweb/?p=gnulib.git;a=b...





Relying on implementation behavior is the perfect way to introduce a hidden in plain site vulnerability.

Most of C++ programs written before P0593R6 depended on implementation behaviour, and were graciously allowed to not be undefined behaviour just 5 years ago. C++ as a language standard is mostly irrelevant, what one should care about is what the compiler authors consider valid code.

Using pragmas, attributes and optimisation guarantees is the point of implementation-defined behaviour in the first place.

The Linux kernel extensively uses gcc extensions. That doesn't inherently make it insecure.


You have to rely on implementation for anything to do with what happens to memory after it is freed, or really almost anything to do with actual bytes in RAM.

Yeah the C committee is wrong here

I don't see why?

The C committee gave you memset_explicit. But note that there is still no guarantee that information can not leak. This is generally a very hard problem as information can leak in many different ways as it may have been copied by the compiler. Fully memory safe languages (so "Safe Rust" but not necessarily real-word Rust) would offer a bit more protection by default, but then there are still side-channel issues.


Because, for the 1384th time, they're pretending they can ignore what the programmer explicitly told them to do

Creating memset_explicit won't fix existing code. "Oh but what if maybe" is just cope.

If I do memset then free then that's what I want to do

And the way things go I won't be surprised if they break memset_explicit for some other BS reason and then make you use memset_explicit_you_really_mean_it_this_time


Your problem is not the C committee but your lack of understanding how optimizing compilers work. WG14 could, of course, specify that a compiler has do exactly what you tell it to do. And in fact, every compiler supports this already: Im most cases even by default! Just do not turn on optimization. But this is not what most people want.

Once you accept that optimizing compilers do, well, optimizations, the question is what should be allowed and what not. Both inlining "memset" and eliminating dead stores are both simply optimizations which people generally want.

If you want a store not to be eliminated by a compiler, you can make it volatile. The C standard says this can not be deleted by optimizations. The criticism with this was that later undefined behavior could "undo" this by "travelling in time". We made it clear in ISO C23 that this not allowed (and I believe it never was) - against protests from some compiler folks. Compilers still do not fully conform to this, which shows the limited power WG14 has to change reality.


Nope it is the C committee

> Once you accept that optimizing compilers do, well, optimizations

Why in tarnation it is optimizing out a write to a pointer out before a function that takes said pointer? Imagine it is any other function besides free, see how ridiculous that sounds?


It's been many years since C compilers started making pathological-but-technically-justifiable optimizations that work against the programmer. The problem is the vast sea of "undefined behavior" — if you are not a fully qualified language lawyer versed in every nook and cranny of the C standard, prepare to be surprised.

Many of us who don't like working under such conditions have just moved on to other languages.


I agree that compilers were too aggressive in exploiting UB, but this is not the topic of this thread which has nothing to do with UB. But also the situation with UB is in practice not too bad. While compilers broke some old code which caused frustration, when writing new code most UB can easily be dealt with in practice by following some basic ground rules (e.g. no unsafe casts, being careful with pointer arithmetic) and by activating some compiler flags. It is not anything that should cause much trouble when programming in C.

Because it is a dead store. Removing dead stores does not sound ridiculous to me and neither is it to anybody using an optimizing compiler in the last decades.

Tree shaking is pretty standard. Optimising out the write sounds fine to me - with the exception of a volatile pointer. That, there, is a mistake.

Optimizing out a write to (example) an array on the stack seems fine to me.

Optimizing out a function call to a heap pointer (especially memset) seems wrong to me. You called the function, it should call the function!

But it's again the C language saving time not wearing a seatbelt or checking the tire pressure for saving 10s on a 2h trip


The whole point of the optimizer is that it can detect inefficiencies by treating every statement as some combination of simple, fundamental operations. The compiler is not seeing "call memset() on pointer to heap", it's seeing "write of variable size" just before "deallocation". For some, optimizing that will be a problem, for others, not optimizing it will leave performance on the table.

There are still ways to obtain the desired behavior. Just put a call to a DLL or SO that implements what you need. The compiler cannot inspect the behavior of functions across module boundaries, so it cannot tell whether removing the call preserves semantics or not (for example, it could be that the external function sends the contents of the buffer to a file), so it will not remove it.


A modern compiler may also completely remove malloc / free pairs and move the computation to the stack. And I do not see what this has to do with C, it should be the same for most languages. C gives you tools to express low-level intent such as "volatile", but one has to use them.

Strong disagree. In C, malloc and free are functions, and I expect no magic to happen when calling a function. If malloc and free were keywords like sizeof, it would have been different.

Your problem is that you're treating words such as "function" and "call" as if they had meaning outside of the language itself (or, more specifically, outside of the C abstract machine), when the point of the compiler is precisely to melt away the language parts of the specified program and be left with a concrete program that matches its behavior. If you view a binary in a disassembler, you will not find any "functions" or "calls". Maybe that particular architecture happens to have a "call" instruction to jump to "functions", but these words are merely homophones with what C refers to as "functions" and "calls".

When you "call" a "function" in the source you're not specifying to the compiler that you want a specific opcode in the generated executable, you're merely specifying a particular observable behavior. This is why optimizations such as inlining and TCO are valid. If the compiler can prove that a heap allocation can be turned into a stack allocation, or even removed altogether (e.g. free(malloc(1ULL << 50))), the fact that these are exposed to the programmer as "functions" he can "call" poses no obstacle.


Closest to what you say that I can find is 5.1.2.3 §4 of N3096

    In the abstract machine, all expressions are evaluated as specified by the semantics. An actual
    implementation need not evaluate part of an expression if it can deduce that its value is not used
    and that no needed side effects are produced (including any caused by calling a function or through
    volatile access to an object)
Problem is, calling external library function has a needed side effect of calling that library function. I do not see language that allows simply not doing that, based on assumed but unknown function behaviour.

You should read "7.1.4 1 Use of library functions". Also "calling a function" is not a side effect.

In C, malloc and free used to often be macros.

As they're freely replaceable through loading, and designed for that, I would strongly suggest that are among the most magical areas of the C standard.

We get a whole section for those in the standard: 7.24.3 Memory management functions

Hell, malloc is allowed to return you _less than you asked for_:

> The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and size less than or equal to the size requested


I read the text as saying the object size can be less or equal to returned memory size. Anyway, section 7 is library. As you say, replacing through loading is a common thing to do — surely compiler is not free to simply elide external library function at will? This is not C++ after all, it must be sensible

If the function is equivalent to a no-op, and not explicitly marked as volatile for side-effects, it absolutely can elide it. If there is a side-effect in hardware or wider systems like the OS, then it must be marked as volatile. If the code is just code, then a function call that does effectively nothing, will probably become nothing.

That was one of the first optimisations we had, back with Fortran and COBOL. Before C existed - and as B started life as a stripped down Fortran compiler, the history carried through.

The K&R book describes the buddy system for malloc, and how its design makes it suitable for compiler optimisations - including ignoring a write to a pointer that does nothing, because the pointer will no longer be valid.


Where exactly does K&R specify the buddy system?

You are literally scaring me now. I'd understand such things being done when statically linking or running JIT, but for "normal" program which function implementation malloc() will link against is not known during compilation. How can compiler go, like, "eh, I'll assume free(malloc(x)) is NOP and drop it" and not break most existing code?



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: