9 minute read
DIGITAL IMAGERY
PIXEL PROPHET WHEN THE MACHINE FAILS — SWITCHES TO MANUAL
Martin Christie has always been a fan of radio, from a time when entertainment from terrestrial television was limited, and on the basis that images created in the imagination by sounds were often more interesting than those projected on the screen. One particular favourite was the original Douglas Adams’ Hitchhiker’s Guide to the Galaxy, in which fantastic events and creatures were generated in the mind which were never really matched by its transition to pictures.
While working from home in lockdown, I listened again to the original series on audiobooks online, and it has lost none of its original magic. The advantage of listening rather than watching is that you can still work on the screen while having one ear on the story.
While browsing that verbal library, and no doubt prompted by algorithms directing me to science fantasy, I came across a short story by EM Forster, The Machine Stops. Better known for novels like A Passage to India, the prolific Edwardian writer penned this tale in 1909, a time when audiences had been thrilled by the written predictions of Jules Verne and H G Wells and are in some ways a reaction to them.
Forster’s future is much darker: an earth where the population has retreated underground to hide itself from a hostile environment and is serviced by a machine that provides all its needs from the air they breathe, the food they eat to the information they receive. As suggested in the title, this will not last forever but has been for so long that the original designers have long died and their knowledge with them. No one recognises that the machine is falling apart, and no one knows how to fix it.
It could have been written today when the Four Horsemen of the Apocalypse have been in full rampage mode, and the climate appears in freefall, not over a hundred years ago in that apparently more carefree world before the First World War.
The great thing about really good science fiction writing is that it has the ability to cast a critical comparison on current events by removing the action and actors to an alternative future - good or bad. Even Adams’ humorous adventures in space and time managed to contain some sharp satires on contemporary life - especially marketing executives!
Regular readers, and indeed most print shop workers, will know that we are subject to customers who rely on a technology they little understand and yet assume to be infallible. Before digital data became mobile, it was more difficult to exchange images, and therefore issues were more easily explained and understood. You almost expected things to go wrong by default.
The reliance on technology has to be measured with human ability to understand it otherwise it does become the demon in the machine that takes over, from Forster’s desolate vision of the future to the smooth but threatening voice of HAL (Heuristically programmed ALgorithmic computer) in Arthur C Clarke’s 2001: A Space Odyssey — “you don’t want to do that, Dave.”
When computers start telling us what we can and can’t do, it’s surely time to check where the plug is. Of course, they already tell us what we like — or what they expect us to like, and we may already be too far down that digital road to press pause but at least hovering a finger over the option to have time to think is better than blindly pressing enter every time.
That very human instinct for caution is why a good part of recent columns has been given over to the increasing use of Artificial Intelligence in the creation and manipulation of digital imaging and why it’s important to understand how it works and why it works the way it does. The more it becomes a part of the normal workflow, the less reason there will be to question it or check its results. And by the same logic, the more you rely on it, the less incentive you have to consider there might be a better alternative.
I even wondered if there be a better name for it if intelligence gave it intellectual credibility it didn’t deserve, but Advanced Logical Interpretation didn’t exactly roll off the tongue. I’m not trying to belittle its obvious capabilities, just put a note to highlight its possible flaws.
I have actually been using a form of AI for over ten years since acquiring an early version of Nik software as a plug-in for Photoshop. This had what Nik called U-point technology which used control points like pins you could use to isolate and adjust colours and shapes, for example, in many clever ways. The smart bit was being able to manipulate specific parts of the picture rather than the whole image, and this also included options for sharpening, reducing noise and the like.
Gradually many of these features, or tools which would perform similar effects, started to be incorporated into Photoshop and then Lightroom, where selections and masks could be edited independently. In addition, the hue, saturation and luminance of individual colours could be diminished or enhanced.
At the same time, Adobe was developing its own AI in tools like Content Aware, first introduced in 2012, to improve upon the potential of cloning and healing of compatible pixels. The brush tools for those tasks are retained for touch-up of parts, but the main selection interface does a serious job of fixing the big cracks seamlessly.
Significantly the working window allows a certain amount of manual adjustment of the controls instead of letting the AI do everything, so you can effectively guide the selection. Similarly, the Select and Mask features and Select Subject and Sky have refining options to make sure the right targets have been recognised.
As I often stress, the software may be intelligent, but it can’t always tell the difference between a bush and someone’s head, and what’s more, it doesn’t care either.
Adobe took a much bigger AI step more recently with Sensei, which is way more than smart image editing but integrating a whole layer of machine learning across the whole creative suite of programmes. The aim is to improve workflow across the board as much time is spent between platforms performing relatively similar, repetitive tasks when creatives should be spending more time on the exciting stuff. At least, that’s the sales pitch. My instincts tell me you still need an eyeball to check results not an algorithm.
The most recent example in Photoshop has been the introduction of Neural Filters as a whole new option in the Filters column. This opens up a new dialogue box with about a dozen new toys you can play with. They
are not installed automatically; you need to download them individually to use them. They are essentially intelligent shortcuts enabling actions that would otherwise take time to do manually and require some experience.
In a recent column, I showed how I used, at least partially, a neural filter to add colour to an old black and white photo, but also illustrated how I had to go in manually to correct some of the assumptions it had made, like not being able to see the difference between grass and gravel on a river bank.
The texture may look different to the human eye, but not that different that it can’t fool a computer, and that is the problem. The machine does need a guiding hand in the first place to recognise subtleties we know instinctively.
Perhaps if it had lots of very similar pictures to edit, it would pick it up, which is the whole idea of AI, but until then, it still has a lot to learn.
Apparently, one of the next neural filters, currently in beta test mode, is picture restoration. I have seen some very mixed reviews on the internet, which doesn’t really surprise me. Although I haven’t been able to download a trial version to test it myself, I can guess it suffers from the same issues.
I do a lot of photo restoration, particularly of people and faces, and there is no quick fix unless you want to just turn the subject into some plasticised cartoon character.
You might be able to get away with a landscape or a building looking a little bit digitised, but a person’s skin, facial features and eyes are a very personal feature, immediately recognisable as human and individual. Our own innate facial recognition ability is still way ahead of any smart camera software.
There are a number of online programs that will happily turn you into a caricature if that’s what you want, but it might not be so appropriate for a funeral display or a special memory! Most people would prefer to see something more authentic and life-like — wrinkles and all. Our facial expressions are sometimes very subtle but can convey a lot of visual information, which is why they are so important, and hard to recreate.
Photoshop has a face recognition feature that can make eyes bigger and a smile wider, but it cannot create a face from a faded photograph.
Old black and white photos fade, losing any deep contrast — the very thing digital needs as a reference, sometimes getting a yellow hue from chemical residue in the paper. Colour tends to lose specific colours first but certainly will tend to lack vibrance and saturation. If things cannot be restored, they have to be reimagined, and that is much more of a human talent than any clever filter can tackle.
A number of these new neural filters work in the cloud - actually using Adobe’s super servers, which presumably are more up-todate than even the latest consumer upgrade, and no doubt also giving valuable feedback on what works and what doesn’t. While all use AI, they obviously apply it in different ways to tackle different tasks but use computer judgement to create the results.
One of the criticisms is that there are not enough adjustment options to control the processes, but maybe this is the point. One way to add some manual intervention is to output the filter as a separate layer and use masks and colour selections as you have done before, for example. So why use the filters in the first place?
Well, like any plug-in, they can be a handy quick fix tool, a handy guide for which way to go - or not, as the case may be. Certainly, with colourising, it seems to work reasonably well on some images and spectacularly badly on others. If you don’t know what the colours should be, it makes a reasonable electronic educated guess. It knows the sky is blue — or anything that looks like sky or grass — for example. I mean, I’m not sure what colour a 1962 BSA sidecar combination was, but this old family group looks pretty good — apart from grannie’s pink nose.
On the other hand, for any accurate colour representation, like those restoration Victorian soldiers in their bright scarlet coats and shiny medals and buttons — forget it. You might recall the magnificent Sgt McGregor from a previous column, a veteran of the thin red line in the Crimean War, wonderfully restored by the patience of Doug Banks down to the finest detail. Not sure the Russians will be quite so in awe of the multi-coloured mess Adobe dressed him in. It looks like the filter just gave up on the challenge. Well, at least I’m not going to be replaced anytime soon by an algorithm!
Incidentally, a set of filters that really does work, the previously mentioned Nik is alive and well and living in Paris, having been taken over and developed by DXO. I have the latest version installed and will show you all its magic next month.