I think everyone agrees that dynamic range compression and de-Bayering (for sensors which are colour-filtered) are necessary for digital photography, but at the other end of the spectrum is "use AI to recognise objects and hallucinate what they 'should' look like" --- and despite how everyone would probably say that isn't a real photo anymore, it seems manufacturers are pushing strongly in that direction, raising issues with things like admissibility of evidence.
One thing I've learned while dabbling in photography is that there are no "fake" images, because there are no "real" images. Everything is an interpretation of the data that the camera has to do, making a thousand choices along the way, as this post beautifully demonstrates.
A better discriminator might be global edits vs local edits, with local edits being things like retouching specific parts of the image to make desired changes, and one could argue that local edits are "more fake" than global edits, but it still depends on a thousand factors, most importantly intent.
"Fake" images are images with intent to deceive. By that definition, even an image that came straight out of the camera can be "fake" if it's showing something other than what it's purported to (e.g. a real photo of police violence but with a label saying it's in a different country is a fake photo).
What most people think when they say "fake", though, is a photo that has had filters applied, which makes zero sense. As the post shows, all photos have filters applied. We should get over that specific editing process, it's no more fake than anything else.
> What most people think when they say "fake", though, is a photo that has had filters applied, which makes zero sense. As the post shows, all photos have filters applied.
Filters themselves don't make it fake, just like words themselves don't make something a lie. How the filters and words are used, whether they bring us closer or further from some truth, is what makes the difference.
Photos implicitly convey, usually, 'this is what you would see if you were there'. Obviously filters can help with that, as in the OP, or hurt.
Another low tech example - those telephoto crowd shots that were popular during covid. The 'deception' happens before the light hits the sensor, but it's no less effective
Pretending that "these two things are the same, actually" when in fact no, you can seperately name and describe them quite clearly, is a favorite pastime of vacuous content on the internet.
Artists, who use these tools with clear vision and intent to achieve specific goals, strangely never have this problem.
But that was the point the OP was making. Not that you couldn’t differentiate between white balance correction and generative fill, but rather that the intent of the change matters for determining if an image is “fake”.
For example, I took a picture of my dog at the dog park the other day. I didn’t notice when framing the picture but on review at home, right smack in the middle of the lower 3rd of the photo and conveniently positioned to have your eyes led there by my dog’s pose and snout direction, was a giant, old, crusty turd. Once you noticed it, it was very hard to not see it anymore. So I broke out the photo editing tools and used some auto retouching tool to remove the turd. And lucky for me since the ground was mulch, the tool did a fantastic job of blending it out, and if I didn’t tell you it had been retouched, you wouldn’t know.
Is that a fake image? The subject of the photo was my dog. The purpose of the photo was to capture my dog doing something entertaining. When I was watching the scene with my own human eyes I didn’t see the turd. Nor was capturing the turd in the photo intended or essential to capturing what I wanted to capture. But I did use some generative tool (algorithmic or AI I couldn’t say) to convincingly replace the turd with more mulch. So does doing that make the image fake? I would argue no. If you ask me what the photo is, I say it’s a photo of my dog. The edit does not change my dog, nor change the surrounding to make the dog appear somewhere else or to make the dog appear to be doing something they weren’t doing were you there to witness it yourself. I do not intend the photo to be used as a demonstration of how clean that particular dog park is or was on that day, or even to be a photo representing that dog park at all. My dog happened to be in that locale when they did something I wanted a picture of. So to me that picture is no more fake than any other picture in my library. But a pure “differentiate on the tools” analysis says it is a fake image, content that wasn’t captured by the sensor is now in the image and content that was captured no longer is. Fake image then right?
I think the OP has it right, the intent of your use of the tool (and its effect) matters more than what specific tool you used.
I don't know, removing the turd from that picture reminds me of when Stalin had the head of the NKVD (deceased) removed from photos after the purge. It sounds like the turd was probably the focus of all your dog's attention and interest at the time, and editing it out has created a misleading situation in a way that would be outrageous if I was a dog and capable of outrage.
The ones that make the annual rounds up here in New England are those foliage photos with saturation jacked. “Look at how amazing it was!” They’re easy to spot since doing that usually wildly blows out the blues in the photo unless you know enough to selectively pull those back.
Often I find photos rather dull compared to what I recall. Unless the lighting is perfect it’s easy to end up with a poor image. On the other hand the images used in travel websites are laughably over processed.
Photography is also an art. When painters jack up saturations in their choices of paint colors people don't bat an eyelid. There's no good reason photographers cannot take that liberty as well, and tone mapping choices is in fact a big part of photographers' expressive medium.
If you want reality, go there in person and stop looking at photos. Viewing imagery is a fundamentally different type of experience.
Sure — but people reasonably distinguish between photos and digital art, with “photo” used to denote the intent to accurately convey rather than artistic expression.
We’ve had similar debates about art using miniatures and lens distortions versus photos since photography was invented — and digital editing fell on the lens trick and miniature side of the issue.
This is a longstanding debate in landscape photography communities - virtually everyone edits, but there’s real debate as to what the line is and what is too much. There does seem to be an idea of being faithful to the original experience, which I subscribe to, but that’s certainly not universal.
There are a whole lot of landscape photographs out there I can vouch for their realism 1% of the time because I do a lot of landscape photography myself and tend to get out at dawn and dusk a lot. There are lots of shots I got where the sky looked a certain way for a grand total of 2 minutes before sunrise, and I can see similar lighting in other peoples' shots as real.
A lot of armchair critics on the internet who only go out to their local park at high noon will say they look fake but they're not.
There are other elements I can spot realism where the armchair critic will call it a "bad photoshop". For example, a moon close to the horizon usually looks jagged and squashed due to atmospheric effects. That's the sign of a real moon. If it looks perfectly round and white at the horizon, I would call it a fake.
News agencies like AP have already come up with technical standards and guidelines to technically define 'acceptable' types and degrees of image processing applied to professional photo-journalism.
You can look it up because it's published on the web but IIRC it's generally what you'd expect. It's okay to do whole-image processing where all pixels have the same algorithm applied like the basic brightness, contrast, color, tint, gamma, levels, cropping, scaling, etc filters that have been standard for decades. The usual debayering and color space conversions are also fine. Selectively removing, adding or changing only some pixels or objects is generally not okay for journalistic purposes. Obviously, per-object AI enhancement of the type many mobile phones and social media apps apply by default don't meet such standards.
I think Samsung was doing what was alleged, but as somebody who was working on state of the art algorithms for camera processing at a competitor while this was happening, this experiment does not prove what is alleged. Gaussian blurring does not remove the information, you can deconvolve and it's possible that Samsung's pre-ML super resolution was essentially the same as inverting a gaussian convolution
And? What algorithm was used for downsampling? What was the high frequency content of the downsampled imagine after doing a psuedo inverse with upsampling? How closely does it match the Samsung output?
My point is that there IS an experiment which would show that Samsung is doing some nonstandard processing likely involving replacement. The evidence provided is insufficient to show that
You have clearly made no attempts to read the original article which has a lot more evidence (or are actively avoiding it), and somehow seem to be defending Samsung voraciously but emptily, so you're not worth arguing with and I'll just leave this here:
I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white):
> somehow seem to be defending Samsung voraciously but emptily
The first words I said were that Samsung probably did this
And you're right that I didn't read the dozens of edits which were added after the original post. I was basing my arguments off everything before the "conclusion section", which it seems the author understands was not actually conclusive.
I agree that the later experiments, particularly the "two moons" experiment were decisive.
Also to be clear, I know that Samsung was doing this, because as I said I worked at a competitor. At the time I did my own tests on Samsung devices because I was also working on moon related image quality
You can upscale a 170x170 image yourself, if you're not familiar with what that looks like. The only high frequency details you have after upscaling are artifacts. This thing pulled real details out of nowhere.
You can try to guess the location of edges to enhance them after upscaling, but it's guessing, and when the source has the detail level of a 170x170 moon photo a big proportion of the guessing will inevitably be wrong.
And in this case it would take a pretty amazing unblur to even get to the point it can start looking for those edges.
I think if you paste our conversation into ChatGPT it can explain the relevant upsampling algorithms. There are algorithms that will artificially enhance edges in a way that can look like "AI", for example everything done on pixel phones prior to ~2023
And to be clear, everyone including Apple has been doing this since at least 2017
The problem with what Samsung was doing is that it was moon-specific detection and replacement
A boss once asked me "is there a way to tell if an image has been Photoshopped?" and I did eventually get him to "yes, if you can see the image it has been digitally processed and altered by that processing". (The brand-name-as-generic conversation was saved for another day.)
> A better discriminator might be global edits vs local edits,
Even that isn't all that clear-cut. Is noise removal a local edit? It only touches some pixels, but obviously, that's a silly take.
Is automated dust removal still global? The same idea, just a bit more selective. If we let it slide, what about automated skin blemish removal? Depth map + relighting, de-hazing, or fake bokeh? I think that modern image processing techniques really blur the distinction here because many edits that would previously need to be done selectively by hand are now a "global" filter that's a single keypress away.
Intent is the defining factor, as you note, but intent is... often hazy. If you dial down the exposure to make the photo more dramatic / more sinister, you're manipulating emotions too. Yet, that kind of editing is perfectly OK in photojournalism. Adding or removing elements for dramatic effect? Not so much.
What's this, special pleading for doctored photos?
The only process in the article that involves nearby pixels is to combine R G and B (and other G) into one screen pixel. (In principle these could be mapped to subpixels.) Everything fancier than that can be reasonably called some fake cosmetic bullshit.
The article doesn't even go anywhere near what you need to do in order to get an acceptable output. It only shows the absolute basics. If you apply only those to a photo from a phone camera, it will be massively distorted (the effect is smaller, but still present on big cameras).
That's just one kind of distortion you'll see. There will also be bad pixels, lens shading, excessive noise in low light, various electrical differences across rows and temperatures that need to be compensated... Some (most?) sensors will even correct some of these for you already before handing you "raw" data.
Raw formats usually carry "Bayer-filtered linear (well, almost linear) light in device-specific color space", not necessarily "raw unprocessed readings from the sensor array", although some vendors move it slightly more towards the latter than others.
In that case you can't reasonably do digital photography without "fake cosmetic bullshit" and no current digital camera will output anything even remotely close to no fake cosmetic bullshit.
That sounds likely. I wonder what specific filters can't be turned off, though. I think you can usually turn off sharpening. Maybe noise removal is built-in somehow (I think somebody else said it's in the sensor).
I think you’ll find that there is no clear line between what you call fake bullshit and the rest of the process. The entire signal path is optimized at every step to reduce and suppress noise. There’s actual light noise, there’s readout noise, ADC noise, often dozens or hundreds of abnormal pixels. Certain autofocus technologies even sacrifice image-producing pixels, and simply interpolate over the “holes” in data.
Regarding sharpening and optical stuff, many modern camera lenses are built with the expectation that some of their optical properties will be easy to correct for in software, allowing the manufacturer to optimize for other properties.
Eh, I'm a photographer and I don't fully agree. Of course almost all photos these days are edited in some form. Intent is important, yes. But there are still some kinds of edits that immediately classify a photo as "fake" for me.
For example if you add snow to a shot with masking or generative AI. It's fake because the real life experience was not actually snowing. You can't just hallucinate a major part of the image - that counts as fake to me. A major departure from the reality of the scene. Many other types of edits don't have this property because they are mostly based on the reality of what occurred.
I think for me this comes from an intrinsic valuing of the act/craft of photography, in the physical sense. Once an image is too digitally manipulated then it's less photography and more digital art.
But when you shift the goal posts that far, a real image has never been produced. But people very clearly want to describe when an image has been modified to represent something that didn’t happen.
I understand what you and the article are saying, but what GP is getting at, and what I agree with, is that there is a difference between a photo that attempts to reproduce what the "average" human sees, and digital processing that augments the image in ways that no human could possibly visualize. Sometimes we create "fake" images to improve clarity, detail, etc., but that's still less "fake" than smoothing skin to remove blemishes, or removing background objects. One is clearly a closer approximation of how we perceive reality than the other.
So there are levels of image processing, and it would be wrong to dump them all in the same category.
For me personally, it's fine to do things like local tone mapping, but object segmentation is where I draw the line. As in, a camera shouldn't know what a sky or a tree or a person is. It shouldn't care about any of that. It shouldn't process different parts of the image differently depending on what it "thinks" is there. Also, denoising should be configurable, because I would always prefer noise over this stupid "painted" look.
ML demosaicing algorithms (e.g. convolutional neural networks) are the state of the art for reversing camera color filters, and this was true back when I did my post-grad studies on the subject almost 10 years ago, not to mention all the other stages of the post-processing stack. So one will have to wrestle with the fact that some form of "AI" has been part of digital images for a while now.
I mean to some degree, human perception is a hallucination of reality. It is well known by magicians that if you know the small region of space that a person is focusing on, then you can totally change other areas of the scene without the person noticing.
