The practice of artificially colorizing photographs dates to the very earliest days of photography in the 1840s, when photographers and their customers would hand paint details on a photograph using semi-opaque dyes of various chemical makeups. Color photography was also attempted in the early days, though with mixed results. Professional, replicable, and commercially viable color photographs would not become mainstream until the 1930s and it would not be until the 1960s that the mass public could afford color photographs.
Artificial colorization of photographs, therefore, was the norm for most people until the 1960s, especially in the late 1800s and early 1900s. Mass market colorization first began with stereographs and large-scale photographs in the 1870s before shifting to postcards in the early 1890s. Postcards themselves were a new medium developed in Germany and Austria as an inexpensive means of sending messages to correspondents without writing a full letter. The practice quickly adopted photographic cards as the primary formula, with a photograph on one side and a space for text and the address on the reverse.
Downsides and problems with colorized photographs:
Artificially colorizing photographs has always been a flawed art. This is partially due to the processes involved and partially from the limitations of the technology. The source photograph was always monochrome—the photographer or an assistant would write down the colors in the actual scene and report those to the printers for them to attempt to reproduce. This, however, left much to be interpreted by the printers, who would have to guess on colors not reported and sometimes ignore notes due to the difficulty of recreating specific colors or shades.
When people wanted to reproduce hand-colorized images, they used chromolithography—or photocrom—a complicated process where layers of color are pressed onto special lithographic limestone stones which are then pressed onto prints, with the colors overlapping until the full color image is created. The depth of color detail on the image is based largely on how many different color stones were used in the process. The process of pressing stones also meant that lower-budget postcards would often ignore color instructions or have misalignments in printing between different colored stones.
Modern technologies introduce their own problems. Additive digital colorization, such as that used by MyHeritage, DeOldify (DeepAI), and other non-generative AI (GenAI) services use an algorithm to compare a monochrome photograph to thousands of pairs of color and monochrome photographs to determine appropriate colorization. This produces mixed results. Colors are often dull and some colors do not render properly, producing bright reds, magentas, and yellows to compensate. This limitation has yet to be corrected in these systems and is unlikely to be fixed due to the mass expansion of GenAI software that can seemingly do the job better. Individual artists also can use software such as Adobe Photoshop to colorize images in a facsimile of a traditional style by hand-drawing colors onto monochrome photographs, thereby creating lifelike colorization that sometimes seems indistinguishable from real color photographs.
GenAI, however, does not colorize a photograph at all. Rather, it recreates the photograph in color pixel-by-pixel. This process always introduces new elements in addition to simple colorization, and therefore it is less a form of colorization and more a type of recreation. This is due in part to programming in GenAI systems that seeks to "restore" and "enhance" photographs, even if that is not the user's desire. It must be emphasized: colorizing a photograph never restores or enhances an image. It is an attempt to make an image appear more lifelike, but the result is only a mimicry of reality. While all colorization is guilty of this, GenAI-produced colorization is far more problematic. The original process, as well as the non-GenAI digital equivalents, involves using a monochrome image as the base image and adding color layers on top of it. GenAI is more akin to painting a scene in color and then presenting it as a photograph.
As a result, artificial colorization in any historical photograph cannot be trusted as a source. It may be almost perfectly accurate, or it could be almost entirely fictional. That doesn't mean it can't be useful, but it must be used with the full knowledge that the colors or objects in the generated image may not be true to life and may be misleading.
Ways of using colorized photographs:
The primary purpose of colorizing photographs is to make history come alive for viewers. It is not a research tool in itself. In fact, the best advice to a historian wishing to use a colorized image in their research is to turn the saturation all the way to 0. Doing this ensures no assumptions are made due to the color. However, using colorized photographs to promote history can be useful so long as it is made clear that the photographs have been artificially colorized and the manner in which they've been colorized. GenAI colorizations especially should always have a watermark or stamp to ensure that the images do not proliferate as actual historical objects, since the internet will not disambiguate on its own.
One exception that sometimes arises is that the colors help make something in an image become more obvious. Granted, this doesn't require color to achieve, but sometimes an automated colorizer such as DeOldify will change the contrast of an image such that something barely visible in the background or washed out by the rest of the image suddenly becomes much more obvious or prominent. This same feat cannot be achieved by using GenAI because GenAI recreates the image, so may simply create something that was not present in the original. But additive colorization can sometimes reveal what was previously not obvious.
Examples of different types of colorizations:
Rather than discussing the matter in the abstract, let's use an actual case study. The photograph below was produced around 1903 by Charles L. Aydelotte's photo studio in Santa Cruz. It is image 59 of a series of photographs, this one depicting a South Pacific Coast Railway passenger train stopped beside the Santa Cruz Union Depot, with a streetcar at the right, several industrial buildings in the foreground, hotel wagons and taxis beside the tracks, and long strings of boxcars in front of the freight depot to the left. The framing was excellent, with the hills and city in the background and a lot of activity around the yard. The photograph was likely taken on top of the station's water tank using a wide angle lens.
![]() |
| Courtesy University of California, Santa Cruz. |
Traditional Colorizing
The above image was so popular that it was colorized at least five times prior to 1920. Examples of these postcards are below, each showing remarkable differences from each other even though they all used the same base image. The only consistent aspect is the cropping: each postcard removes the bottom quarter inch of the photograph, thereby removing the original Aydelotte caption and studio signature.
Example 1
This example does a classic technique of intricately tracing the contours of a dark feature in a lighter color, in this case touching up the locomotive, to emphasize its natural lines. Otherwise, this photograph is probably the most accurate of the examples, depicting more realistic colors for the buildings and trains, albeit with a stronger usage of red than would probably reflect reality (see, for instance, the almost pink Santa Cruz High School in the left distance). A new postcard series has been added to the sky, noting "S. C. 122. Depot, Santa Cruz, Cal."
Example 2
This example retains the largest portion of the original image, only cutting off a little of the bottom. However, to remove the original captions, a hand-drawn bush has been placed in front of the shack to the right to cover up the Aydelotte studio signature and the roof and tracks on the bottom left have been carefully colorized to hide the original caption. Instead, a new caption has been written below the photograph: "Birdseye View from Union Depot, Santa Cruz, Cal." Note this isn't, in fact, a bird's eye view. The colors trend toward the bluer side of the spectrum, possibly the result of deterioration or simply a lower quality print. All of the colors are more prominently drawn, further suggesting a lower budget printing.
Example 3
Obviously this example suffers from an excited postcard user using every available space to write their message. Yet some features are still very apparent. The image in this postcard is framed by a thick white border with the publisher on the left side and "Southern Pacific R.R. Station, Santa Cruz, Cal." written on the bottom. This is a very low budget three-tone colorization, with the printer only using blue, yellow, and red to create all of the colors. It is also washed out so much that specific features are indiscernible. The use of red for roofs is a very common colorization technique when the true colors are not known, while green for all foliage is a natural fall-back. Like the first two examples, the train cars are all shades of red. The bottom and top of the original image have been cut back, the bottom more than the top.
Example 4
This is a more extreme, though lifelike, colorizing that reverses the trend of the previous example by rendering most of the roofs in blue. This is another lazy fall-back when true roof colors are unknown. The rolling stock are unremarkably yellowish green or brown rather than the more vibrant red of the examples above. The boxcars especially almost blend into the greenish-yellow ground. Other features jump more, including the red cowcatcher on the train, the very white steam, the turquoise blue sky, and the many shades of color in the foliage. The buildings, except their roofs, are also uniquely drawn, including those far in the distance on the hills. This postcard's colorizer took shortcuts, but also put some effort in their work. It is part of a long series, probably with an international audience in mind, numbered "549 Union Railroad Depot, SANTA CRUZ, California."
Example 5
And last of the original postcard colorizations, this example does everything wrong. Like Example 4, it relies on only three colors, this time cyan, magenta, and yellow. The result, however, is jarring. Unnatural reds jump out where there should be natural browns. The roofs are all slate gray. The greens are dark. The yellows appear beige. And the sky is a sickly greenish-yellow. The contrast of the original photograph has also been increased in a possible attempt to make the image easier to colorize on a budget. The result is less than ideal, though, and proves that original colorizing techniques were not always successful. The postcard is fairly common so was probably intended for visiting tourists and may have been a domestic production. Its caption, "R.R. Station, Santa Cruz, Cal.," is also the simplest of this series of original postcards.
Additive AI colorization
The next two examples are modern colorizations created using AI machine learning models that add a color layer on top of the original image. Because this method is non destructive, these examples are closer in relation to the colorized postcards above than to the generative AI models that follow. However, the colorizing is created using an algorithm rather than any human input. As explained earlier, this method is produced by taking a color photograph, desaturating it (removing the color), and then training the AI model on what colors are produced from which shades of gray and common features compared to the original color image. In other words, it is a system intentionally designed to add colors to monochrome photographs in a way very similar to the method of hand colorizing photographs. Of the two examples below, DeOldify is an older iteration of that used by MyHeritage, yet the development of the tool from 2018 to 2021 is remarkable, though neither model is without major flaws. Unfortunately, the rapid expansion of GenAI-based colorization has led all developers to drop this earlier form of AI-driven colorization despite its many benefits compared to entirely generated colorized photographs.
DeOldify (DeepAI)
This earlier colorization model was still fairly basic and suffered because of that. The reds and yellows were overly emphasized to the detriment of the colorization. Even the train has a yellowish hue to it and the entire image appears sickly. Greens, meanwhile, have bled out of the foliage and onto surrounding features, such as the roof of the structure in the foreground. And there are still spots of gray around, such as behind the word "Santa Cruz" in the photo caption. The photo studio credit, meanwhile, has been turned red, which appears to be a general failure of the colorization algorithm since this often occurs in photographs colorized using DeOldify-based colorizers. The boxcars are dark and largely colorless, and most of the buildings in the backgrounds have red roofs and white walls. Although it is better than some of the hand-drawn colorizations above, it still leaves a lot to be desired.
MyHeritage
The creator of DeOldify sold his design to MyHeritage in 2020 and it was refined for two years before MyHeritage abandoned the technology in favor of GenAI "enhancement." The old technology is still available, though, and creates somewhat more lifelike colorizations than its predecessor. However, the results are mixed, such as in the example above. The entire scene looks like it has a gray filter and had its contrast increased, washing out the colors. Greens are generally good and the ground is passable, but the buildings and vehicles are overly brown and red, with unrealistically bright reds on the roofs of the buildings at left. While it doesn't suffer from the extreme colors of the DeOldify colorization, it swings too far the other way by not inserting any vibrancy in the colorization at all.
Generative AI colorization
The following images were generated using five different GenAI models using the prompt: "Colorize this circa 1903 commercial photograph realistically, as if it were a photograph taken with a modern color camera." This is a relatively simple, openly-interpreted prompt, much like the type used by most people creating GenAI colorized photographs. The results are remarkably different, though. While some features such as the sky being blue, foliage being green, and the ground being brown-tan are relatively consistent, almost everything else is rendered differently even though they appear nearly identical at a glance. Even the blue sky features entirely different cloud patterns and types, despite the fact that no clouds at all are discernible in the source image.
ChatGPT
The image generated by ChatGPT produced a vibrant scene. It has removed the captions, and all of the text except "Santa Cruz" on the depot is indiscernible. The scale of the barn at right is slightly smaller than in the original, and the roof is not quite the same. Similarly, the small shack in the foreground is smaller and its chimney is shorter and fatter. The hills in the background are mostly the same shape but more forested in the ChatGPT render. An extra story has also been added to the high school and there are several differences in the buildings and clearing to the left of the boxcars. The lamp at the top of the telephone pole in the center of the image is also slightly different than the one in the original source photograph.
Firefly
Adobe's GenAI model, Firefly, takes a more vibrant approach, rendering the train's passenger cars in a bright burgundy, except for the first car for some reason. Meanwhile, the boxcars are all in dark brownish-reds creating a rather muddy scene. The locomotive belches out black smoke and white steam, the former of which is not hinted at in the original photo. The foreground in the Firefly render looks much more true to the original image, and the coaches, wagons, and people feel more active than in the ChatGPT render. Even the background looks better, although the buildings to the left of the boxcars seem a little too clean and tidy compared to the likely reality. While there are minor deviations in the image from the source, they are relatively minor, the most obvious of which is the disappearance of the large light from the top of the telephone pole at the center of the image.
Gemini
Google's Gemini AI engine has become integrated into Google search via its AI Overview summary that often appears at the top of Google search results. Its image generation, though, leaves much to be desired when it comes to generating colorizations. While the foreground appears to be mostly accurate to the original, including the captions, the colors across the board seem unrealistic. The locomotive is multicolored and belching smoke rather than steam. The cars are a forest green, and the depot—including its eaves—is mustard yellow. The boxcars, though, appear more realistic, with some in different colors and most in the characteristic brick red. Again, the text on the buildings and vehicles cannot be read. The hills are surprisingly yellow in this render, contrasting it with the more meadowy greens of the previous two renders. The background buildings are a spectrum of colors, which probably don't correspond to reality, though may at least be in the spirit of the times. Changes from the original photograph include the addition of more information around all four sides, a reduction of steam/smoke from the locomotive, resulting in several more boxcars appearing, and the loss of the large light at the top of the telephone pole in the middle of the scene.
Grok
Grok, developed by SpaceX, generated a more subdued image, darker than the others as if a cloud is hovering over Santa Cruz. Unlike the renders above, the train is better colored with dark roofs, a solid black locomotive except for its front, and brown paneling on the cars. The depot also looks more natural, even though the words are unreadable on the station sign or on any of the surrounding vehicles. The boxcars are stereotypically reddish, and the locomotive's steam is appropriately white. The hills, too, show a rather natural blend of greens and yellows. The AI tool has added a little more of the image to the left, showing another building and completing the boxcar. The buildings in the background, however, are all rather dull shades of gray and blue, and there is a general lack of definition in the render when zoomed in, most notably with the white building to the right of the high school in the distant left. That being said, it is the most accurate of the models in retaining fidelity to the source photograph.
EzEnhancer
Lastly, there is EzEnhancer, a free-use image "colorizer" not run by any of the major tech companies. Unsurprisingly, the colorization is the worst of the GenAI models, with weird tone changes such as on the barn on the right of the render and the multi-colored boxcars. The locomotive and its steam appear accurate, but the cars are a dark brownish-gray. The ground around the tracks seems more gray than tan, which sounds right but looks wrong. The tool has expanded the image slightly on all sides except the right, adding another matching house to the five in the source photograph. More generally, the greens are a little too vibrant, the reds a little too magenta, the ground a little too gray, and the whole image just feels flat. The people are also poorly rendered and, as usual, all of the text is unreadable. Even the caption is a little off, with the "y" in Aydelotte smaller and less recognizable than in the source photograph.
Generative AI video colorization and animation
As one final note on the topic of colorization, GenAI models are also able to animate still photographs. This has the great potential of making history come alive, but can easily be misused. A good video render also requires a lot of patience to properly program to ensure everything in the video makes internal and contextual sense. Both of these examples below have obvious problems.
Video 1: Firefly
With just the simple prompt of "animate this circa 1903 photograph of the Santa Cruz train station," Firefly decided to zoom in on the train idling at the station, belching out copious amounts of steam and smoke while people chat loudly beside it. The sound effects of the train are non-existent and the added musical notes attempt to create a feeling of tranquility. Even when the AI zooms in, the text on the depot and wagons remains illegible. While the people are moving around a little, the horses aren't, nor are the wagons, and the entire scene feels slightly unnatural.
Video 2: DeeVid AI
This video used the same prompt as above and used the ChatGPT image as its source. It's produced using a free GenAI video rendering tool. The result is a much more vibrant scene. The train actually leaves the station (surprisingly fast!) and people are walking around as it departs. The sound effects are that of a train leaving, so feels more appropriate to the scene. The sign on the station is readable as "Santa Cruz" even though none of the other text can be read. The camera moves slightly, giving the feel of a real photographer filming the scene. However, like the Firefly video, none of the horses move and some of the people who are moving are doing so strangely.
Conclusions
In the end, all of these colorization methods introduce problems. While traditional colorization and additive colorization do not harm the photograph usually, they also often get key colors wrong in sometimes distracting and misleading ways. Meanwhile, Generative AI potentially introduces a range of problems since it completely recreates the original image pixel by pixel, always changing a few things along the way. In most of the cases above, that was limited to adding more image around the sides, changing the light on a telephone pole, and rendering text into gibberish, but other GenAI renderings have made far more extreme mistakes including introducing entirely new elements into the image. GenAI colorization, however, does often look more realistic, though, partially because the rendering process allows for more accurate color ranges, not reliant on the original shades of the monochrome photograph, and partially because GenAI-created images are sharper and more naturally contrasted than other colorization methods.
Thus, the matter really comes down to each user and whether they prefer to retain the integrity of the original image or are fine with rendering an entirely new version inspired by the original. The important thing to remember is to always mark modern artificially colorized photographs using a stamp, watermark, or signature on the image so that people recognize that it is a modern recreation. Colorization is more popular than ever before, with films, documentaries, and commercial businesses using various technologies to bring color to the past. It is the responsibility of historians, artists, and colorizers to ensure this trend does not accidentally lead to confusing historically colorized images with those produced in modern times.
Further reading:
- Alicia J. M. Colson, "Colorizing Historical Photographs: Ethical Dilemmas of a New Digital Technique," First American Art Magazine (Spring 2024).
- Elisa Mariarosaria Farella, "Colorizing the Past: Deep Learning for the Automatic Colorization of Historical Aerial Images" Journal of Imaging (October 1, 2022).
- Gwen C. Katz, "A Critical Evaluation of Colorization APIs," Medium (December 30, 2024).
- Thomas Nicholson, "YouTubers are upscaling the past to 4K. Historians want them to stop," Wired (October 1, 2020).
- Roshaya Rodness, "The controversial history of colorizing black-and-white photographs," Fast Company (May 20, 2021).








.jpg)
.png)
.png)
.jpg)
.jpg)

No comments:
Post a Comment
Note: Only a member of this blog may post a comment.