WO2000017818A1

WO2000017818A1 - Method for processing digital images by means of a computer

Info

Publication number: WO2000017818A1
Application number: PCT/NL1999/000585
Authority: WO
Inventors: Igor Alphons Asselbergs
Original assignee: Kodia V.O.F.
Priority date: 1998-09-21
Filing date: 1999-09-21
Publication date: 2000-03-30
Also published as: NL1010137C2; AU6009799A

Abstract

The method relates to processing, by means of a computer, digital images of objects such as, for instance, buildings and motorcars, which are recorded in their natural surroundings. The method comprises the following steps performed by means of the computer: 1) from an original image of the object, at least one portion to be processed of the object is selected; 2) at least the portion to be processed of the object is displayed on a display of the computer and retouched such that said portion of the object looks like it is painted white, while during retouching, the color of the light from the surroundings of the object falling on the portion to be painted white is taken into account; 3) a transparent color is laid over the white-retouched portion of the object for coloring the selected portion of the object, the thus colored and processed portion being displayed on the display; and 4) the portion of the object selected in the original image is replaced by the portion colored in step 3, the thus obtained processed image being displayed on the display.

Description

Title : Method for processing digital images by means of a computer.

The invention relates to a method for processing, by means of a computer, digital images of objects such as, for instance, buildings and motorcars, which are recorded in their natural surroundings. The invention also relates to a method for displaying by a computer an image of which at least one portion has been processed.

Processing, by means of a computer, digital images of objects such as, for instance, buildings and motorcars is known per se. Such operation involves a photo of the object being digitized and fed in digitized form to the computer for further processing. If one subsequently wishes to change colors on a photographed object by means of the computer, a number of problems are encountered. In this context, "color" is understood to mean a specific combination of brightness, saturation and tone. For instance, dark objects on a photo often contain too little information to be able to render them light. This can be explained from the fact that shadows on a light object on a photo can become so dark that they are virtually black. This black forms a substantial contrast with the white parts of the object on the photo. On the other hand, the black of the shadow of a dark object has only little contrast with the rest of the object. Hence, in practice, the object on a photo which is rendered light by means of a computer often becomes quite flat and spotty.

Further, it is a fact that the computer can change the color of a specific part of the photo in such a manner that all color dots, or pixels, of the photo in that part obtain a specific saturation and tone. In other words, these pixels for instance each acquire the same color red, while the brightnesses and, accordingly, the shadows in the portion to be edited of the photo are preserved. In other words, the relative brightness of the pixels relative to each other is preserved. However, it turns out that if the image of the object is edited in this manner, the object comes to look particularly unnatural. For instance, a building displayed on a photo" and whose color is adjusted in the known manner may look particularly unnatural and stand out, as it were, from its background. A viewer gets the idea that the building consists of an image of a drawn building pasted on a photo of the surroundings of the building. Hence, a problem of the known method is that the coloring of an object in a digitized image produces a highly unnatural impression. The purpose of_ the invention is to provide a solution to this problem.

Accordingly, the method according to the invention is characterized in that it comprises the following steps performed by means of the computer:

1. from an original image of the object, at least one portion to be processed of the object is selected;

2. at least the portion to be processed of the object is displayed on a display of the computer and retouched such that this portion of the object looks like it was painted white before the image is made, while during retouching, the color of the light from the surroundings of the object falling on the portion to be painted white is taken into account ;

3. a transparent color is laid over the white-retouched portion of the object for coloring the selected portion of the object; and

4. the portion of the object selected in the original image is replaced by the portion colored in step 3, the thus obtained processed image being displayed on the display.

The invention is based upon the insight that for processing the image of the portion of the object by means of the computer, the influence of the light from the natural surroundings falling on the object portion to be processed can only be properly taken into account when the processing operation is split up into a number of steps. As in step 2, the object is retouched by means of the computer as if it was painted white, the color of the light from the surroundings of the object falling on the portion to be retouched white can be properly taken into account. In step 3, only the color of the portion to be processed of the object itself has to be taken into account. If, for instance, the object stands outside at sunset, the incident light will be colored slightly red. The white-painted portion of the object will reflect this red light and will hence also be colored slightly red itself. During retouching, this can be properly taken into account, because red incident light will give the_ white-painted object a red glare. Accordingly, for instance blue incident light from the blue sky falling on another white-painted portion of the object will give the portion in question a bluer glare. When the incident light from the surroundings of the object is thus properly taken into account during retouching in step 2, a color can subsequently be added to the selected portion in step 3 in a manner known per se. In principle, only the color of the light falling on the object determines the manner in which the portion is to be retouched in order to look white. Hence, in step 2, only the color of the light from the surroundings needs to be taken into account. As a result, in step 4, there is obtained a processed image which produces a completely natural impression. If one wished to take into account the incident light according to the prior art, this would involve a great many problems. If, for instance, the object should be colored blue in one step, it is highly difficult to take into account the color of, for instance, the red-colored light from the surroundings in addition to the color of the object itself. After all, it is not directly clear how both color effects combine and form a total image. As it turns out, by separating both effects from each other, with in step 2 only the color of the incident light from the surroundings being taken into account, while in step 3 the color of the object itself is added, a completely natural image is obtained. Preferably, it applies that before steps 3 and 4 are performed, the portion of the object selected in the original image is replaced by the portion retouched in step 2, the thus obtained processed image being displayed on the display. In this ^"manner, it can be seen and estimated directly whether the selected portion of the object has been properly retouched white, taking into account the incident light. In particular, it applies that in this manner, the result of ^"the white-retouching of the selected portion of the object can directly be viewed on the display.

The editing of the selected portion of the object can be performed by means of image-processing programs known per se. A drawback of these image-processing programs, however, is that they only show the result of the processed portion of the object. The purpose of the invention is to meet this drawback as well. Accordingly, a particular variant of the ^• method according to the invention is characterized in that the image obtained in step 3 is stored in the computer as a first image; a second image is stored in the computer comprising the original image; and that in step 4, the first and second images are jointly displayed, such that the retouched portion of the image and the original image less the portion of the original image retouched in the first image are displayed in combination to form one uninterrupted image. Thus, the result of the image-processing of the selected portion, in combination with the rest of the original image, can be viewed directly.

This part of the method can of course also be applied without the processing operation of the selected portion of the object comprising at least steps 2 and 3. Instead, steps 2 and 3 can also be replaced by one step according to which the selected portion comprises another type of image processing. Accordingly, the invention also relates to a method for displaying an image by means of a computer, of which image at least one portion is edited by means of a computer, this method being characterized in that a first image is stored in the computer in which only the processed portion of the original image is visible: that a second image is stored in the computer comprising the original image; and that the first and second images are jointly displayed on a display ^" connected to the computer, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image.

In particular, in the last-mentioned method, a selected transparent color can be laid over said portion of the first image. This portion of the image can then correspond to said first portion of the object.

The invention will presently be specified with reference to the accompanying drawings . In these drawings :

Fig. 1 shows an image of an object, in this case a house, located in its natural surroundings; Fig. 2 shows a first number of portions to be processed of the object;

Fig. 3 shows a second number of portions to be processed of the object;

Fig. 4 shows the portions of Figs. 2 and 3 that have been retouched white;

Fig. 5.1 shows the original image;

Fig. 5.2 shows the original image, in which the"Object portions to be processed of the original image have been retouched as if they are painted white; Fig. 5.3 shows the image according to Fig. 5.2, in which the white-retouched portions have undergone further color processing;

Fig. 6 shows a computer 6 with a display with which the method according to the invention can be performed; Fig. 7a shows an original image of a building;

Fig. 7b shows the image according to Fig. 7a processed in a manner known per se; and

Fig. 7c shows the image of Fig. 7a processed according to the invention. Hereinafter, with reference to Figs. 1-5, a possible embodiment of the method according to the invention, to be performed by means of a computer, will be discussed. Fig. 1 shows a house 1 placed in its natural surroundings. The image of the house in its natural surroundings is digitized. The digitization can be performed by means of a scanner known per se . It is also possible that the photo according to Fig. 1 is made by a digital camera, so that the image is digitized directly. Anyhow, the digitized image is eventually fed to a computer 2 of Fig. 6. The computer 2 consists of a PC known per se which, moreover, comprises a color screen 4. Further, the computer is loaded with an image-processing program known per se, such as Photoshop. However, image-processing programs other than Photoshop may be applied as well.

The starting point is that in this example, insight must be gained as to what the house 1 looks like if the horizontal wooden panels fixed on the outside of the house are painted green. Hence, this requires processing the image of Fig. 1 in such a manner that the panels are green, to create a good impression of what the house looks like when the house with such green panels is placed in its natural ^' surroundings .

For performing the method by means of the computer, the horizontal panels 6 are selected first, being the portions of the object 1, in this case the house 1, which are to be processed. Selecting these portions can be effected with the above-mentioned known image-processing program. Fig. 2 shows the portions 6 of the house 1 which have thus been selected in a first processing step. Next, in a second step, the portions to be processed are retouched in such a manner that these portions of the object look as if they are painted white, while during retouching, the color of the light from the surroundings of the object falling on the portions to be painted white is taken into account. The image in which the selected portions are retouched white is displayed on a display (see Fig. 5.2) . In this manner, it can be directly assessed whether during white-retouching, the color of the incident light from the surroundings has been properly taken into account. The white-retouching of the selected portions of the object and displaying the original image in which the selected portions have been replaced by the white-retouched portions can again be performed by the image-processing programs known per se.

In particular, it applies that during the performance of step 2 by means of an image-processing program known per^¬ se, the object portion selected in the original image is replaced by the object portion processed during step 2, the thus obtained processed image being displayed on a display. In this manner, during white-retouching, the result thereof can be assessed directly on the basis of the image in its entirety. The portion of the object retouched in step 2 is stored in the computer as a first image. Also, a second image is stored in the computer comprising the original image. Next, these first and second images are jointly displayed by means of a separate computer program, which differs from the program whereby the selected portions have been white- retouched, in such a manner that the processed portioπ^~of the original image and the original image less the portion 6 of the original image processed in the first image are displayed in combination while forming one uninterrupted image. In this context, an "uninterrupted image" is understood to mean a continuous image without staggering. This is again shown in Fig. 5.2. Hence, briefly stated, Fig. 5.2 shows the house with its horizontal panels retouched white, while, moreover, these panels are retouched white in such a manner that the influence of the light from the surroundings is taken into account. In this example, light from the surroundings is rather gray-tinted, as a result of which the panels, viewed objectively, are retouched gray yet appear to be white on the photo. Subsequently, in a third method step, a transparent color is laid over the white-retouched portion of the object in a manner known per se by means of the above-mentioned separate program, for coloring the selected portion of the object. The portion thus colored and processed is again displayed on the display. In particular, the portion is displayed directly in combination with the rest of the house (Fig. 5.3) . According to the invention, this last is effected in that, in a manner known per se, by means of said separate program, the image obtained in step 3 is stored in the computer as a first image. Hence, this concerns an image as shown in Fig. 2, in which, however, the selected portions of Fig. 2 have the green color as shown in Fig. 5.3. Also, the above-mentioned second image is stored in the computer which comprises the original image. Next, the first and second images are jointly displayed in method step 4 by means of the separate computer program, in such a manner that the processed portion of the image and the original image less the portion of the original image processed in the first image are displayed in combination while forming one uninterrupted image as shown in Fig. 5.3.

In this manner, there is obtained an image of a'house in which a portion of the house is provided with another color in such a manner that it produces an entirely natural impression. The color of, in this case, the horizontal panels of the house is determined by the chosen color of the panels themselves as well as by the color of the light from the natural surroundings of the house 1 falling on these panels. The eventual result proves to correspond particularly accurately with reality. This particularly good and natural result can be achieved precisely because the panels are retouched white first, with the color of the incident light being taken into account. After all, the white-retouched portion of the house will reflect the color of the incident light, with the color of the incident light being identical to the color of the reflected light. This would be entirely different if, for instance, the relevant portion were already painted green. In that case, during retouching, it is difficult to feel with what color the incident light is reflected by the colored object. Hence, the above implies that in step 2 exclusively the color of the light from the surroundings falling on the selected object has to be taken into account, and that in step 3 only the chosen color of the selected object has to be taken into account. The combined result yields a particularly natural effect. Moreover, it is a fact that the process of laying a selected color over the white-retouched portion is known per se and can readily be performed by means of the computer.

In particular, it applies that during the performance of step 2 by means of an image-processing program known per se, the portion of the object selected in the original image is replaced by the object portion processed during step 2, with the thus obtained processed image being displayed on the display. In this manner, during white-retouching, the result hereof can be assessed directly on the basis of the image in its entirety.

In digitized photos, straight lines fairly often prove to acquire a stepped character. To prevent this, an anti- ^■ aliasing can be performed in step 2 as well. This may again be carried out by means of the known image-processing program. Also, it is possible that in step 1, by means of the image-processing program known per se, a percentage of p% is selected from at least a number of pixels of the selected portion of the object 1, wherein p is less than 100%. The selected portions can be stored in shades of gray, wherein white stands for p=0% selection, black for p=100% selection and wherein the shades of gray represent values wherein it applies that 0<p<100. Each processing operation performed on these pixels by means of this selection will only have p% effect. Hence, it is for instance possible to select the pixels of the door 8 for 50% for further processing. It is also possible that in step 2, damaged parts of the selected portion are also retouched out of existence. This, too, can be performed in a manner known per se by means of the image- processing program.

According to the invention, it is also possible to select different portions of the house to cause them to undergo a different treatment. As shown in Fig. 3, the windows and doors 8 can also be selected as a number of portions of the house 1 to be processed separately. These portions can first jointly be retouched white, as shown in Fig. 4. Fig. 4 then forms a first image which can be displayed (not shown) together with the second image by means of the separate program in a manner known per se. Subsequently, in a first third step, the panels can be colored with a first color, and in a second third step, the door and the windows can be colored with a different color.

This, too, can be performed in a manner known per se by means of the separate program. Such variants each fall within the framework of the invention. Of course, it is also possible that the selected portion of an object comprises the entire object. In this regard, one may for instance think of a bridge which is to be painted blue in its entirety. In that case, by means of said known image-processing program,-- the bridge itself is selected from an image of the bridge in its natural surroundings. Next, by means of the computer, the selected bridge is retouched in such a manner that the entire bridge looks like it has been painted white, while during retouching, the color of the light from the surroundings of the bridge that falls on the bridge to be painted white is taken into account. The whole, including the surroundings of the bridge, is visible on the display. The thus white- retouched bridge or already partially white-retouched bridge is stored again in the computer as a first image. Next, a second image is stored in the computer comprising the original image. Also, by means of the separate program, the first and the second image are displayed in combination in a manner known per se, so that the processed bridge and the original image less the bridge are displayed in combination while forming one uninterrupted image. Under the first image, a selected transparent color can again be laid by means of the separate program, in a manner known per se . It is also possible to selectively display the original second image or said combination of the first and the second image on the ^" display of the computer. This enables the original image to be properly compared with the processed image. This part of the method can also be understood by starting from the idea that the first image lies "at the bottom" and is thus invisible, while the original image lies "on top". Next, the first image is made visible in that in the original image, lying on top, the portions to be processed are left out.

With reference to Figs. 7a, 7b and 7c, another example of the embodiment of the method is illustrated. Fig. 7a shows the original image of a portion of a building. In respect of this building, it should be considered in what manner the wall, built from bricks, and the horizontally and vertically extending concrete piles, and the vertical panels connected to these piles, can be colored differently from each other.

Fig. 7b shows the effect when the wall is painted yellow, the vertical concrete piles are painted blue, the horizontal concrete piles are painted dark-green and the panels are painted light-green. Here, the influence of the color of the light from the surroundings is not taken into account. The fact that prior to the painting, the stones will be cleaned is not taken into account either, because fouling of the stones is still visible. Fig. 7c shows the final result of the method according to the invention. Here, the horizontal concrete piles are provided with a new color. Also, damages have been retouched out of existence. The horizontal concrete piles form the first selected parts of the object and are retouched blue. The wall forms the second selected portion of the object which, according to the invention, is retouched yellow, while, moreover, damages and fouling are removed.

Third selected objects are formed by the horizontal concrete piles which are now retouched dark-green. Finally, the vertical panels form fourth objects, retouched light-green. Moreover, damages are removed. Also, at the top edge of these panels, it is clearly visible that an anti-aliasing treatment has been carried out in step 2 of the method.

Each of the above-mentioned image treatments can be performed by means of image-processing means known per se . However, it is not easy to provide a total picture of the image obtained in step 4 by the image-processing means known_. per se. Hence, the invention also relates to a method for meeting this problem. The method has already been discussed with reference to the examples given hereinabove . This method in fact implies that the displaying, by the computer, of an image of which at least one portion has been processed is effected as follows: a first image is stored in the computer in which only at least one processed portion of the original image is visible; a second image is stored in the computer comprising the original image ; and - the first and the second image are jointly displayed on a display connected to the computer, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image. When used in the method outlined hereinabove, the portion of the image corresponds to the portion of the object. Optionally, the entirely original image and the combination of the above-discussed first and second images can be displayed alternately. Also, a selected transparent color can be laid over said processed portion of the first image, to create a new first image. This new first image can again be displayed together with the second image, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image. It is also possible that said first and

Claims

second images are displayed in combination as soon as said portion of the original image is processed with said transparent color. After all, the result of the processing operation is directly visible on the display. It is also possible that a number of different portions are included in the first image, while different transparent colors are laid over different portions in a manner known per se. Such variants are each understood to fall within the framework of the invention.CLAIMS

1. A method for processing, by means of a computer, digital images of objects such as, for instance, buildings and motorcars, which are recorded in their natural surroundings, characterized in that the method comprises the following steps performed by means of the computer :

2. at least the portion to be processed of the object is displayed on a display of the computer and retouched such that said portion of the object looks like it was painted white before the image is made, while during retouching, the color of the light from the surroundings of the object falling on the portion to be painted white is taken into account; 3. a selected transparent color is laid over the white- retouched portion of the object for coloring the selected portion of the object; and 4. the portion of the object selected in the original image is replaced by the portion colored in step 3, the thus obtained processed image being displayed on the display.

2. A method according to claim 1, characterized in that before steps 3 and 4 are performed, the portion of the object selected in the original image is replaced by the portion retouched in step 2, the thus obtained processed image being displayed on the display.

3. A method according to claim 1 or 2 , characterized in that in step 1 the pixels of at least one portion of the selected portion of the object are selected for a percentage of p%, wherein p<100.

4. A method according to claim 3, characterized in that after step 1 has been performed and before step 2 is performed, the selected portion is stored in the computer in_. shades of gray.

5. A method according to any one of the preceding claims, characterized in that during the performance of step 2, the portion of the object selected in the original image is replaced by the portion of the object retouched white during step 2, the thus obtained processed image being displayed on the display.

6. A method according to any one of the preceding claims, characterized in that in step 2 an anti-aliasing is carried out as well.

7. A method according to any one of the preceding claims, characterized in that in step 2, damaged parts of the selected portion are retouched out of existence as well.

8. A method according to any one of the preceding claims, characterized in that the selected at least one portion of the object comprises the entire object.

9. A method according to any one of the preceding claims, characterized in that the object in the original image is placed in its natural surroundings.

10. A method according to any one of the preceding claims, characterized in that the image obtained in step 3 is stored in the computer as a first image; a second image is stored in the computer comprising the original image; and that in step 4, the first and second images are jointly displayed, such that the retouched portion of the image and the original image less the portion of the original image retouched in the first image are displayed in combination to form one uninterrupted image .

11. A method according to any one of the preceding claims, characterized in that the image obtained in step 2 is stored in the computer as a first image; a second image is stored in the computer comprising the original image; and that the first and second images are jointly displayed, such that the retouched portion of the original image and the original image less the portion of the original image retouched in the first image are displayed in combination to form one uninterrupted image .

12. A method according to claim 10 or 11, characterized in that selectively, the second image is displayed on the display or the first and second images are jointly displayed, such that the retouched portion of the original image and the original image less the portion of the original image retouched in the first image are displayed in combination to form one uninterrupted image .

13. A method according to claim 10, characterized in that the first and second images are jointly displayed, such that the retouched portion of the original image and the original image less the portion of the original image retouched- in ^• the first image are displayed in combination to form one uninterrupted image instead of the original image as soon as said portion of the original image is processed with an image-processing program known per se .

14. A method according to any one of the preceding claims, characterized in that the retouching in step 2 is performed by means of an image-processing program known per se, such as Photoshop.

15. A method for displaying, with a computer, an image of which at least one portion is processed, characterized in that a first image is stored in the computer in which only the processed portion of the original image is visible; that a second image is stored in the computer comprising the original image; and that the first and second images are jointly displayed on a display connected to the computer, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image .

16. A method according to claim 15, characterized in that^" a selected transparent color is laid over the processed portion of the first image for coloring the portion of the first image .

17. A method according to claim 15 or 16, characterized in that selectively, the second image is displayed on the display, or that the first and second images are jointly displayed, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image .

18. A method according to claim 16, characterized in that the first and second images are jointly displayed, such that the processed portion of the original image and the original image less the portion of the original image processed in the first image are displayed in combination to form one uninterrupted image instead of the original image as soon -as the particular transparent color is laid over the first image, said portion of the original image being processed with an image-processing program known per se .

19. A method according to any one of claims 15-18, characterized in that the first image is produced from the original image by means of an image-processing program known per se, such as Photoshop.