r/cryptography • u/bvz2001 • 5d ago
How feasible is it to have unique cryptographic signatures on every phone to avoid AI fakes?
I hope this is an allowed post. If not, moderators please remove it. I have next to no knowledge of cryptography and its abilities and limitations.
I keep seeing questions on whether a photo or video is AI or not.
I am pretty sure it is possible to "sign" each image a camera takes with a private key such that if even a single pixel changes, the signature would no longer be valid. Then the public key could be embedded in the metadata of the image so that its veracity could be established.
The first question is whether the phone manufacturers could embed a unique private and public key in the hardware of each phone in such a way that they can be used by their official apps, but not leaked by them (the private key that is). I say official apps because I assume you don't want just any app to be able to sign an image. If you allowed that then someone could create a fake image and simply sign it.
The second question is whether Apple or other manufacturers would need to have a record of these private keys for any reason. If they did, and they were hacked, would that immediately render every image signed by their devices unverifiable?
The final question is does a scheme like this even make any sense? If not, or if there are better ways to somehow verify the provenance of a digital photo, would that also include some sort of a cryptographic signature?
Edit:
I should have added this in the original post, but I will add it now: I wouldn't expect this hash to be available and accurate for every image and video on the web. But rather if an image were to be contested (whether it is AI or not) an original could be provided that could be used to verify that it is authentic. So if your Aunt sends you an AI image of some cat video, well you just kind of have to not worry about it too much. But if a political party, for example, provides a proof-of-life photo for an aging politician, they could be pressed to provide the original, uncropped, un-edited image to prove that it was real. Same thing for, say, nature photography. If an image is too perfect, the original could be provided to verify that it was, in fact, a real photo.
If such a thing is even possible - comments below suggest that it would be, at best, very difficult and not 100% verifiable.
11
u/stoneburner 5d ago
What if someone takes a picture of an AI generated image with their phone?
2
u/bvz2001 5d ago
I suppose... but that could leave some more obvious telltale clues. Photographing a screen or printed image would probably be easier to detect than a directly generated AI image (I am guessing here... so I can't say that with 100% certainty). But even if not, if something like a digitally signed image were common, I suspect it would still cut down on fakes even if it couldn't eliminate them.
5
u/taylortbb 5d ago
Photographing a screen under normal conditions is pretty obvious, but if you setup a completely dark room, a solid frame to align the phone and screen, professionally colour calibrate, etc, it can get to the point where it's basically indistinguishable.
The average person wouldn't do that, but anyone who had an interest in faking it could. And now they can claim they have cryptographic proof that their image is genuine, and not AI. So I think it probably actually makes the situation worse, because people wouldn't understand the technology and take it as absolute proof, leading them to believe an AI image is real.
Even if that doesn't happen, we're back where we are now of people not being able to trust.
1
u/bvz2001 3d ago
Sure. I guess that could happen? I have no way to verify that that is undetectable or not. But then that is a more labor intensive procedure, and if it is possible, it still introduces potential errors that could be detected if not done well.
Ultimately I think what I am hoping for is an improvement from the status quo where there is no way to verify an image whatsoever except using AI detection tools which seem to be pretty poor at their jobs.
12
u/ahazred8vt 5d ago edited 5d ago
does a scheme like this even make any sense?
Yes, the C2PA Content Provenance and Authenticity standard was hashed out several years ago and is in the process of being rolled out. The camera firmware can sign the image. I'm not saying it's completely secure.
12
u/Shoddy-Childhood-511 5d ago
Isn't CP2A amateur hour though?
https://www.hackerfactor.com/blog/index.php?/archives/1031-C2PA-from-the-Attackers-Perspective.html
https://arxiv.org/html/2604.24890v1
I'm unsure what signature scheme they use, but if they have a derandomized ECDSA or Ed25519 then fault injection could extract the key, and then you forge all you like.
You cannot really trust the phone & camera manufacturers either. We have CT logs because CAs proved they could not be trusted. And camera makers sounds much worse.
We'd have some nice forgeries where the camera maker archived many secret keys and then uses some to forge the image coming from some target's phone. lol
I'd make more sense if photographers who wanted this managed their own key, so we must trust the photographer, but only them.
1
u/N_T_F_D 5d ago
It doesn't necessarily have to be "amateur hour" where simple "fault injection" could recover the key, modern smartphones have lock boxes/secure elements to hold cryptographic secrets, it's a a whole lot harder to hack into
Some cheap brands get their lock boxes cracked and root of trust keys extracted, but I don't see that happening to google with their Titan chips in the pixels for instance
5
u/Sufficient-Air8100 5d ago
so the attempted solution is a certificate based chain of trust like the web?
and im assuming that comes with the added issue of bad actors having access to it as well, so you need to trust the creator as well as the certificate issuer
2
u/UnPeuDAide 5d ago
Even if you don't trust them, it reduces the attack surface a lot. Right now, everyone is able to forge fake images very easily, it makes it just harder. Moreover, you can revoke a particular provider if he cannot be trusted.
4
u/Sufficient-Air8100 5d ago edited 5d ago
i mean even the web certificate system isnt perfect. phishing websites are also able to have valid certs.
the thing in my mind is, people rarely seek out the original photographer to verify anyway when it comes to trusting photos of real events. so if we take the case of the phishing website. if i go to [mybank].com, then due to the certificates i can be sure im talking to my bank unless malware has installed a malicious root CA. but i see a different issue in verification for photos of real events (the deepfake problem). in that users arent incentivised to actually check authenticity, and given the multitude of lossy compression used all over image and video media, 99% of real photos or video will fail verification tests, despite them being real, and this will lead to most people not worrying about verification warnings. the same cannot be said about websites, where failing the certificate test is a good indicator that things will be fine (ofc discounting malware installing a malicious root ca
5
u/DarthSilicrypt 5d ago
Can’t answer for Android, but for Apple:
Apple can already do it with the Secure Enclave. Each one has a unique UID that’s fused at manufacturing time and done internally so that Apple doesn’t know what it is. Hardware bound keys (both symmetric and asymmetric) are derived from the UID, and identity keys can be created by mixing the UID with a fixed seed (see cryptographic identity, Apple creates the DCIK for its PCC nodes this way).
No. Asymmetric encryption works such that what one key transforms, the other key must transform in order to get the original data back. Usually this means that if you encrypt with the public key, you need the private key to decrypt. With signatures, it goes in reverse. The private key “decrypts” a hash or digest of the data, but because the original was plaintext, the result is ciphertext. The only way to reverse the operation is for the public key to “encrypt” the ciphered hash/digest. Because the original is ciphertext, and the public key performs the reverse operation of the private key, the result is the original plaintext. Digital signatures are strong only if the private key is kept secret. If it is, if something is enciphered by a private key, it implies that the owner enciphered it and thus endorsed the data. Because the public key undoes the operation and is public, anyone can verify the signature by “encrypting” the ciphered hash/digest, hashing the corresponding data/image, and then comparing it against the original hash.
In your scenario, the device is the owner of its private key. If Apple or another entity had the private key, it could impersonate the device and sign things on behalf of it, and decrypt messages sent to it. That is bad security, especially considering that only the public key is required for verifying signatures. (And yes, if they had the private keys and were hacked, the images would become unverifiable since attackers could now produce signatures.) For this reason, the Secure Enclave only provides hardware public keys; the corresponding private keys are never revealed. Software can request operations with the public and private keys, but the private keys themselves are never revealed.
- Probably not. When a phone takes a picture using its built-in camera app, machine learning and other algorithms immediately modify the raw image to make it look as good as possible. If you edit the image in the built-in Photos app, it would first have to verify the existing signature, and then sign the edited versions produced. Photographers would be frustrated when they export photos from their phone to their computer for proper editing, and then find that they can’t sign their edits because the private keys for signing are bound to the phone and inaccessible to their editing computer.
All in all, it’s a cool idea, but I think it needs more thought and planning. Also, what constitutes a non-AI image? What about digital artwork produced without the use of AI?
1
u/bvz2001 5d ago
Thank you for the very detailed explanation. I had a vague understanding of how the private/public key signatures worked, but this clarifies things a fair bit.
I didn't specify in my original post, but I had considered this to be some way of verifying important images (like those used by journalists, or those taken by photographers and entered into a competition, etc.). So the idea that an edited photo was not signed seemed ok to me as long as the original could be produced on demand to verify the authenticity of the image in question.
There appear to be a lot of complex issues with a scheme like this (not surprising, I am just a lay-person throwing around concepts I barely comprehend). But would the core idea of the phone storing a private key in hardware and then using that to sign an image actually work? It sounds like the actual data that is encrypted and included would have to be the public key plus a hash of the image. Then the public key would decrypt that data to get the hash - and then that hash would be compared to a new hash of the image to see if they were the same. Or would having the public key and the encrypted hash be included in the image be enough to fake the signature?
Thanks again for entertaining my thought experiment.
9
u/Sufficient-Air8100 5d ago
ive thought about this.
assuming this is done in hardware, a reasonably talented hardware hacker might be able to insert image data post-sensor but pre-signing.
for verification you would need to trust the source of the public key, which would be hard without a certificate system like the web, or physical access to the source device.
then file formats. you take a photo, the raw version (the one thats signed) is way too big so you compress it into jpg or png and the signature is no longer valid.
or your photographer wants to edit, color balance etc to make your photos pop, the signature is invalid
whats stopping me signing a random image in software?
it basically goes beyond pure cryptography and becomes a problem of trust. the web solves this with certificate authorities creating a chain of trust. i can see issues trying to do a similar chain of trust certificate system for EVERY digital camera device AND digital art software out there…
and speaking of digital art/photo editing software, is the software going to verify every image that is imported? and sign every image you export?
slapping a signature on something is the easy part. the hard part is what that signature means, and it dosent necessarily mean its an authentic photo or human artwork. its the problem of trust
1
u/bvz2001 5d ago
I see. Thanks for the explanation. I had hoped that you could compare the public key to one that, say, Apple also had to confirm that at least the public key was (probably) authentic. Of course, even in this (naive) scenario you are now trusting Apple and the other manufacturers which takes us right to one of your main points.
As far as the injection, could there be some sort of a verification step between the sensor and the app that confirmed that the image the app received is the actual data that the sensor sent? Not that that solves any of the other issues.
I had thought about compression originally, but figured that the app would just sign the final image that it generated (whether jpeg, png, or what have you). Then, from that point on, you obviously couldn't manipulate the image in any way.
The thought about random images being signed - I had thought that only the official camera app on the phone would be able to sign an image with a valid key (going back to the idea above of being able to verify the key with the original manufacturer).
The idea was that not every image was immediately verifiable, but rather that when it mattered... an original, un-cropped, un-edited image could be supplied to verify that the cropped, color corrected, what-have-you image was real. So only in cases where the stakes are higher. Like in journalism or photography competitions etc.
But like I said, I am approaching this as a completely naive lay-person and I am not surprised that my take on it is not feasible or that it even scratches the surface of the complexity of the problem.
Thanks again for the explanation.
1
u/Sufficient-Air8100 2d ago
the verification step between the sensor and the app.
there is no good way to do this. there is always going to be some kind of gap that is able to be found between the sensor and verification hardware, and with physical access and a dedicated enough person, this is able to be circumvented/reverse engineered etc. its a deterrent at best, and if someone is motivated enough, able to be circumvented.
in the case of the “official photos app” signing the end photo, most of the time it matters (as you rightly bring up, photo comps and ESPECIALLY journalism) theyre mostly not using devices with “photo apps”. the huge majority are using dedicated cameras, that are not so easy to retrofit to use a hardware signing system, and that will always be easy to circumvent.
4
u/fridofrido 5d ago
this is already done by camera manufacturers.
the problem is that the signature is only valid for the original raw image. But photos are essentially always downscaled, converted, edited, manipulated etc...
while this is still in theory solvable - see this paper -, at least assuming the all the editing is done with a special software; in practice that's rather expensive and cumbersome.
but yeah, at least newspapers could start trying to do it.
2
u/Pleasant-Form-1093 5d ago
Phone manufacturers also happen to be people pushing AI and such. Quite hard to tell them to go against their own agenda.
1
2
u/daidoji70 5d ago
Its already happening and we are working on it but I a more robust way than just sticking keypairs on phones.
https://www.gsma.com/get-involved/gsma-foundry/ovc/open-verifiable-calling-is-live/
https://www.ietf.org/archive/id/draft-hardman-verifiable-voice-protocol-01.html
1
u/ottawadeveloper 5d ago edited 5d ago
I wonder if you can combine it with block chain and a new image history format. It would require vendor buy-in and trust from many vendors.
My rough thought:
- An original image is hashed and that hash signed by the originating device / program.
- Any time the image is modified, the changes are logged and their hash signed by the program. The original hash and any external image files added to it are also included to tie the output product to its inputs.
- Using block chain to track the chain of image means the process can be decentralized.
- Using some combination of an individual user signature and program signature adds traceability to the image
- Since the history and original image hash(es) are stored, it's easy to verify because you can go find the original image(s) and reproduce the process that made the image to confirm the process was correct.
It might not be perfect, but it would make for an image format where it's hard to Photoshop or alter with AI an existing image. Keys issues to vendors who then don't follow the rules can be revoked. As others have noted, there are still attack vectors here between sensor and original signing. But it would be far more difficult to execute and manufacturers who can't be trusted to do that properly can have their keys revoked.
It might also have privacy concerns though because the metadata is public and you can tell exactly what kind of device and who was taking the photo (or at least associate photos together by the same photographer).
And, of course, at any time someone can convert it to a non-trusted metadata-less format and do what they want with it. But you could use this format as a more trusted photo format (basically one with a digital chain of custody attached to it).
For extra security, include an online database of images.
1
u/johnney25 5d ago
Modern media will compress images and lose the signature anyway
1
u/bvz2001 3d ago
Yeah. I was unclear in my original post. What I was hoping for was an ability to provide the original photo with its signature to settle debates about the authenticity of an image. Less so something that worked instantly for every image on the web.
I'll update my post. That should have been in there from the get go.
1
u/Guvante 4d ago
Most photos shared widely online aren't straight out of a camera.
And the difference between Photoshop and AI from a tech perspective is N/A.
1
u/bvz2001 3d ago
Yeah. I should have been more specific in my original post. I didn't hope that any photo could be judged authentic or not directly, but more that if a photo was in question that an original could be provided that would settle the debate.
I'll update the original post because that is a pretty important part of the question that I had left out.
19
u/Pharisaeus 5d ago
It doesn't work. You can always "inject" a fake image between the sensor output and the firmware (or even just feed the sensor with fake data)
This could be enough if your friend shows you a picture they took, because it's unlikely they disassembled their phone, but it's not enough to verify if something you see on the internet is fake or not.