![\"\"](\"https:\/\/jimhphoto.com\/wp-content\/uploads\/2024\/03\/wizard-of-oz_curtain.webp\")
<\/figure>\n<\/div>\n\n\n\n\n\n\nI worked in software development. Neural nets aren’t new – they were invented in the 1940s, but didn’t go anywhere because there wasn’t enough computing power and data storage for them to do much. Over the decades, AI hype came and went, research papers were written, computers played chess and wrote poetry, and the public wasn’t really all that impressed. But now the hardware is catching up with the software and surprising new things are possible.<\/p>\n\n\n\n
After years of generating as much excitement as a dead raccoon on a hot highway, “artificial intelligence” hype is running wild. Pundits say it will wipe out entire job categories. Highly leveraged Silicon Valley execs tell baffled judges and politicians that AI systems “learn like a child” and warn them not to lay down in front of a train by trying to regulate it. And talentless wannabe artists are “crafting prompts”, generating attention-grabbing mages and representing them as their own “work”.<\/p>\n\n\n\n
So what’s really going on? Did something major just happen, and is “generative AI” based on a big breakthrough that I hadn’t heard about? I wanted an honest, non-technical explanation of how something like Stable Diffusion, Midjourney or DALL-E works<\/strong>. Here is the 2 minute summary <\/em>of what really went on at Stability AI <\/strong>(the company behind Stable Diffusion):<\/p>\n\n\n\n They started with a “publicly available dataset” containing “5 billion image-text pairs” – that means images and associated text taken from web sites like Pinterest, WordPress blogs, DeviantArt and Flickr.. (Roughly 700k came from Fine Art America, where I sell photos, so I’m probably in there somewhere.)<\/p>\n\n\n\n Note that “publicly available” just means “publicly accessible”, and that these images were used without regard for copyright, permission, privacy or attribution<\/em> – so no surprise that Getty Images is now suing Stability AI for a trillion dollars.<\/a> The web-scraping was done by a “non profit” called Common Crawl, giving Stability AI a fig leaf of cover for a massive misappropriation.<\/p>\n\n\n\n The images were then “filtered” (AI term) to decompose them into visual elements software can work with – think lines, shapes, edges, textures and colors. This process, analogous to turning raster images into vectors, is the basis for claims by execs and lawyers that the images weren’t “copied”.<\/p>\n\n\n\n Next, the heavy lifting: a neural network build a vast set of correlations between those filtered elements – visual similarities, and statistical correspondence with identifying words – according to set of human-created rules called “weights”. It’s a huge computing task, involving enormous amounts of data, and neural networks are indeed powerful new tools for this work. (To really learn something about neural nets, see this excellent post<\/a> by Stephen Wolfram.) The end product is a sort of gigantic database that can do one thing: produce an image of something specified by text. That image will be a clever amalgam of a subset of those millions of images lifted from the web. And the text associated with these images ( embedded key works, ALT text ) is the only way AI “knows” what the images contain.<\/p>\n\n\n\n The magic is in all those “weights”, essentially the rules governing how image elements will connect and relate to each other – where do they come from? This part gets deep and is obviously highly proprietary, but weights are assigned by humans, with the intent of getting a result they like, and to some extent it’s trial-and-error on a very large scale. In the end, it’s whatever works. <\/p>\n\n\n\n But wait – the resulting images are beautiful, right? Doesn’t this imply knowledge of the principles of aesthetics and design? Didn’t the AI “learn” this somehow? Not quite. The correlations in Stable Diffusion were also weighted by the inclusion of another data set called “LAION-Aesthetics Predictor”, which was itself created by showing large numbers of images to humans, who scored them for visual appeal.<\/p>\n\n\n\n This process – collecting our images and our descriptive without permission, extracting the content, finding the correlations and similarities in that content, and ranking its aesthetics using algorithms based on human reactions, is basically what’s called “training a model”.<\/em><\/p>\n\n\n\n The system now has the ability to generate an image based on text input – if<\/em> the model contains images matching that text – and make it superficially similar to images with high aesthetic scores. Given the size of the input data, practically everything in the world is in there, so you’ll get some<\/em> sort of image as output. And here’s where an AI model really departs from a relational database: because the neural net builds connections based on relative similarity, it can now generate seemingly “new” images, which are really combinations of the contents of some of the originals used in the the training. Think of these generated images as existing “in between” the human-created originals, and combining the content of their neighbors.<\/p>\n\n\n\n So there you have it: a grossly unfair over-simplification of an enormously complicated software system. But I think it captures the essence.<\/p>\n\n\n\n You may be thinking: “wait a minute, where’s the intelligence<\/em>? All I see is other people’s content, reduced to numeric data, correlated according to predetermined rules, indexed by keywords, and mashed up in clever ways.” You’re not wrong. Generative AI is just an information processing system, totally passive; it produces an image in response to a prompt, from human-created content that’s been linked to those keywords. The image won’t look exactly like any of those used to create it, although it may be darn close<\/a>.<\/p>\n\n\n\n Now, consider this<\/strong>: the process of “training a model” is very expensive<\/em> in terms of computing power, bandwidth and storage; it’s not going to happen every time someone requests an image. In fact, the interval between refreshes of that data could be very long – possibly years. Especially if the AI company is meanwhile tied up in court, defending itself against accusations of copyright violation.<\/p>\n\n\n\n And this:<\/strong> so-called AI has absolutely no visual input other than the work of humans. It’s never “seen” anything in the real world, and it won’t, until it gets a swarm of camera drones. And even then it won’t know what anything actually “is” until humans tell it.<\/p>\n\n\n\n So we could starve it to death<\/strong> – well, hypothetically – if we all removed all our content from the internet next week. Or, just stopped adding any new images, period. These marvelous “generative AIs” would then be stuck in 2024 forever<\/em>, unable to update their models. And in a few years, their output will be seriously out of date and won’t look so awesome anymore…<\/p>\n\n\n\n Generative AI is a massive appropriation of the work of real artists and photographers.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[654],"tags":[542,780,791,787,790,786,789,792,788],"post_folder":[],"class_list":["post-4817","post","type-post","status-publish","format-standard","hentry","category-thoughts-on-photography","tag-ai","tag-artificial-intelligence","tag-copy","tag-copyright","tag-creative","tag-generative","tag-neural-net","tag-non-technical","tag-stable-diffusion"],"_links":{"self":[{"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/posts\/4817","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/comments?post=4817"}],"version-history":[{"count":0,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/posts\/4817\/revisions"}],"wp:attachment":[{"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/media?parent=4817"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/categories?post=4817"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/tags?post=4817"},{"taxonomy":"post_folder","embeddable":true,"href":"https:\/\/jimhphoto.com\/index.php\/wp-json\/wp\/v2\/post_folder?post=4817"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
This Wikipedia article<\/a> is a good starting point. But reality still hides behind jargon like “denoising autoencoders” and “multimodal transformers”, while ethical responsibility is obscured with weasel words about “research”, “nonprofits” and “publicly accessible data sets” – and some very creative interpretations of copyright law.<\/p>\n\n\n\n![\"\"](\"https:\/\/jimhphoto.com\/wp-content\/uploads\/2024\/04\/70s-3.jpg\")
<\/figure>\n","protected":false},"excerpt":{"rendered":"