A C T A U N I V E R S I T A T I S L O D Z I E N S I S
F O L IA O E C O N O M IC A 175, 2004
J e r z y K o r z e n i e w s k i *
M E T H O D O F C O L O U R S E G M E N T A T IO N IN I’W O D IM E N S IO N A L IM A G E S
Abstract. The paper is divided into tw o parts. In the first part an overview o f som e selected m ethods o f segm enting colours in tw o dim ensional im ages is given. In the second part a new algorithm is proposed. The new algorithm is different from other algorithm s due to its stress on accuracy o f colou r classes, the smallest possible number o f colour classes (conditionally on param am eter choice) and due to smaller stress on the small number o f eventual segm ents. T h e algorithm perform ance is assessed through app lication s to the segm entation a couple o f colourful images.
Key words: colou r segm entation, EM algorithm, non-param etric density estim ation, pixel clustering.
I. IN T R O D U C T IO N
Im age segm entation is a basic task in such areas as im age search in m ultim edia libraries, m edical p h o to s analysis, object recognition in in du stry, ro b o t co n tro l and m an y others. Searching th ro u g h the W eb pages and th ro u g h lite ra tu re one can find a n u m b er o f m eth o d s which were invented to deal w ith the problem : pixel based techniqnes ( P a u w l e s , F r e d e r i x 1999), area based techniques ( S k a r b e k , K o s c h a n 1994), edge-detection ( G e v e r s , G h e b r e a b , S m e u l d e r s 1998), physics-based segm en tatio n ( K l i n k e r , S h a f e r , K a n a d a 1990), n o n -p a ram etric co lo u r density estim ation, EM alg o rith m based on m arginal colo u rs d istrib u tio n ( D e m p s t e r , L a i r d , R u b i n 1977). All o f these m ethods have their own advantages and d isadvan tages o r features, as one should call them m o re ap p ro p riately . The tro u b le with co m p arin g these m eth o d s w ith one a n o th e r lies in the problem o f the choice o f criterion. M o st o f the m eth o d s were developed with a view to m eet differen t criteria. T h o se w hich give ac cu rate co lo u r segm entation are no t fast enough and vice versa, those which give reasonable
(in m ost ap p licatio n s) n um ber o f colo ur segm ents d o n o t define co lou rs precisely enough an d so on. Below som e exam ples o f ap p licatio n s o f som e m eth o d s m en tio n ed above are given and intuitively and visually assessed.
In Fig. 1 there is an exam ple o f applying the algo rithm based on non-param etric colour density estim ation to the segm cntatien o f an artificially crcatcd im age co n sistin g o f a circle an d a b a c k g ro u n d o f “ s m o o th ” tran sitio n betw een qu ite different colours (e.g. blue an d yellow). T h e m ain idea o f the m eth o d is to estim ate the density o f co lo u rs (suitably defined) with the form u la
M = C ^ K ' i g - g,)t i
W here g t d enotes level o f grey o f pixel i, С is a certain c o n sta n t, and the kernel used was a G au ssian
W hen th e co lo u r density was estim ated it was possible to “ hill clim b” in the list o f k n o t p o in ts (the one seen in the m iddle picture) in o rd e r to find a local peak for each d a ta point. T h a t is why the result is com posed of tw o segm ents because linking any pixel with its local co lo u r peak is the feature o f the alg o rith m , therefore pixels o f qu ite different colo u rs can be linked th ro u g h g ra d u al steps.
Fig. 1. A n exam ple o f applying algorithm based on non-param etric density estim ation to the segm entation o f an image with sm ooth transition between colours
Fig. 2. A n exam ple o f applying EM algorithm to the segm entation o f a city’s landscape
In Fig. 2 a typical exam ple o f applying EM algorithm to the segm entation o f a city landscape (it was o btained at h ttp://ciips.ее.w illiam s.uw a.edu .a n / ~ ). T he m ain idea o f this algorithm is to present the density o f th e m arg in al d istrib u tio n o f the intensities o f grey in the form o f the sum o f local densities w hich are supposed to be know n th ro u g h assum ed p rio r densities. T hen the coefficients o f this presentation are estim ated (E -step) an d , next, the posterio r p rob abilities for each pixel being o f each co lo u r are calculated and each pixel is assigned to its m ost likely colour (M -step). T he segm entation o f the b lack -and-w hite p h o to results in a couple o f shades o f greyncss (six or seven) co rresp o n d in g , m ore o r less, to d ark er and b righ ter frag m en ts o f the image. M o re o r less is a p ro p e r phrase because n o t all frag m en ts are correctly assigned to colour.
2. N E W ALG O R ITH M P R O PO SA L
A new algorithm we w ant to p ropose consists o f a n u m b er o f stages, each o f w hich consists o f a n u m b er o f steps. T h e n u m b er o f stages is no t co n stan t and d epends on the precision o f segm entatio n we w ant to achieve. T he steps o f each stage are the following:
1) establishing best n eig h b o u rh o o d s for each pixel, 2) linking n eig h b o u rh o o d s w ith sim ilar colours, 3) sep aratin g a n u m b e r o f initial colours,
4) establishing best n eig h b o u rh o o d s for the colo urs sep arated ,
Below, we give a sh o rt description o f each step. In the first step we visit each pixel in tu rn and we look for the best n eig h b o u rh o o d o f this pixel in the sense o f the largest neigh bourh oo d consisting o f pixels w hose colours arc n o t fu rth e r aw ay from the colour o f pixel i th an the value o f param eter p a ri. T h e d istance o f the colours o f tw o pixels is the sum o f absolute values o f differences in the levels o f red, green an d blue betw een the levels o f these three basic colours, in th e colou rs o f the tw o pixels being com pared. T h e succession o f pixels being co m p ared d epend s on the type o f n eig h b o u rh o o d we consider. A ltogether, there are five different types as show n in Fig. 3. T ype 3 is type 2 ro tated 90 degrees aro u n d pixel with n u m b er 1, an d type 5 is type 4 transfo rm ed in the sam e way.
1 90 73 74 75 76 91 89 72 40 41 42 52 77 92 71 39 23 24 25 43 53 78 93 70 38 22 14 10 15 26 44 54 79 69 37 21 9 2 3 16 27 55 80 68 36 13 8 1 4 11 28 56 81 67 35 20 7 6 5 17 29 57 82 66 48 34 19 12 18 30 45 58 83 65 47 33 32 31 46 59 84 94 88 64 49 50 51 60 85 95 87 63 62 61 86 96 2-3 51 39 40 41 42 43 44 56 52 38 24 14 15 16 17 25 45 53 37 23 13 6 1 2 7 18 26 46 36 22 12 5 4 3 8 19 27 47 35 34 21 11 10 9 20 28 48 54 50 33 32 31 30 29 49 55 4 -5 36 37 38 35 19 20 39 34 17 18 21 40 33 16 8 12 22 41 32 15 2 3 9 23 31 14 7 1 4 24 30 11 6 5 25 29 13 10 26 28 27
A fter assigning to each pixel th e largest possible n eig h b o u rh o o d (with established co lo u r i.e. levels o f red, green and blue) we link sim ilar colo u rs i.e. we create g ro u p s o f colours, the co lo u r o f each g ro u p is th e arith m etic m ean co lo u r o f the colo u rs o f all n eig h b o u rh o o d s n o t fu rth e r aw ay from the co lo u r o f the initial neighb o u rh o o d (which is th e first largest neigh b o u rh o o d in the list o f all neighb o u rh o o d s) th an the p a ram eter par2 value.
A fter linking co lo u rs o f neighbo u rh o o d s in to groups o f colo u rs we arrive at a certain n u m b er o f colours (usually very large) and we perform step 3 by choosing first n colours in the list o f all colours. E ach o f the colours chosen is fu rth e r from all o th er colours chosen th a n twice the num ber p a r i -I- par2.
S eparation in step 3 is do n e in ord er to avoid o verlap ping o f co lo u rs in step 4 w hich is m ad e in a sim ilar way as step 1. T h e only difference is th a t n eig h b o u rh o o d s consist o f pixels belonging to one o f the co lo u rs separated in step 3. In step 5 we rem ove (e.g. set co lo u rs o f the pixels already assigned to be negative num bers) from the im age pixels w hich have been assigned to any o f the neig h b o u rh o o d s established in step 4 and we can sta rt next stage carried o u t in identical way. T h e n u m b er o f stages needed to assign all pixels to som e colours is usually big (a cou ple o f dozen) for the sam e values o f p a r i and par2 th ro u g h o u t th e alg orithm . H ow ever, a b o u t 90% o f all pixels can be assigned to colo u rs in the first couple o f stages (five o r six).
T h e n u m b er o f stages and th e num b er o f colou rs defined in each stage obviously depends the choice o f p aram eters p a r i and par2. H ow ever, it tu rn s o u t, th a t these values can n o t be to o small, because it results in to o m any sim ilar colo u rs, and they c a n n o t be to o big, because it results in generating new colo u rs, n o t present in the original im age (while defining new colours th ro u g h arith m etic m eans). F o r a general so rt o f im age one can use values p a r i = 8 an d par2 = 5. S egm entation is alm ost the sam e for p aram eters values sm aller o r greater by one.
3. A L G O R IT H M A PPLIC A TIO N
T o see hew the alg o rith m perform s let us apply it to a general so rt o f im age like e.g. a p h o to g ra p h o f a building seen in Fig. 4. T h e results o f each o f six stages are show n in Fig. 5 and th e co lo urs received in each stage are given in T ab . 1. E ach picture in Fig. 5 corresp o n d s to one stage and con tains only pixels assigned to colours in this and earlier stages. T h e segm entation w as p erfo rm ed for p aram eters p a r i = 8 an d par2 = 5.
Fig. 5. R esults o f first six stages o f segm entation o f a castle’s im age with the new algorithm (first three in the upper row and last three in the bottom row)
T a b l e 1 C olours defined in each o f the six stages
Stage I Stage 2 Stage 3 Stage 4 Stage 5 Stage 6
R О li R О 1) R G B R G B R G B R G B 217 225 224 227 233 235 222 229 228 230 236 238 232 232 222 120 146 122 172 193 196 212 220 219 206 216 215 228 227 215 110 140 118 105 97 63 200 212 212 134 208 209 234 239 242 216 217 202 131 148 127 106 133 107 237 242 244 175 183 164 188 193 173 159 170 152 222 223 210 214 222 223 4 28 33 165 190 193 224 221 204 194 205 203 87 106 82 171 144 84 0 3 18 60 65 42 188 206 208 94 112 89 207 209 193 215 212 192 18 15 22 180 198 200 170 176 159 119 136 113 142 160 141 79 90 65 28 45 38 63 42 30 34 29 28 182 186 166 195 130 68 88 101 76 201 201 181 195 198 173 74 83 57 145 157 134 114 129 102 136 155 134 186 203 203 160 168 145 128 142 120 57 72 52 87 90 58 91 120 93 166 174 154 140 152 130 80 33 24 205 204 185 148 87 45 192 123 59 35 61 53 153 163 140 72 92 73 82 71 49 178 179 155 98 118 96 77 49 36 165 171 147 198 209 207 46 53 39 197 195 172 186 181 161 148 163 144 16 39 42 97 47 25 174 196 200 192 191 167 4. C O N C L U SIO N S
M any practical app licatio n s o f the algorithm propo sed allow us to assess it in the follow ing way.
1. I he algorithm is different from o th er algo rith m s w ith respect to p u ttin g m ore stress on accu rate segm entation o f colo u rs and less stress on the small n u m b er o f segm ents;
2. 1 he algorithm is practically unsupervised because one can alw ays use values o f p aram eters p a r i = 8 and par2 = 5 unless one needs very fine differentiatio n betw een colours and is n o t both ered with th o u san d s o f segm ents which result; in such situ atio n s one should use sm aller p aram eters;
3. 1 he colo u rs defined by the algorithm are well defined i.e. th ere are no very sim ilar tw o colours;
4. I h e algorith m is relatively slow, each stage takes ab o u t 10 m inutes on a 900 M H z co m p u ter, b u t 90% o f this tim e is w asted fo r reading and w riting d a ta on disc; therefore the whole tim e could be m u ch sm aller if softw are with bigger o p eratin g m em ory were used.
Jerzy Korzeniewski
R EFEREN CES
D e m p s t e r A. , L a i r d N. , R u b i n D . (1977), M a xim u m L ikelih o o d fr o m Incom plete Dala Via the E M A lgorithm , “ Journal o f the R oyal Statistical S ociety” , Series B, .49, 1-38. F a n g Y ., T a n A. T . (2000), N ovel A daptive Colour S egm entation A lgorithm a n d its
Application to S k in D etection, Proc. BM VC, 23-31, BM VA.
F i n l a y s o n G ., T i a n G . (1999), Colour N orm alisation f o r Colour O bject R ecognition, “ International Journal o f Pattern Recognition and Artificial Intelligence", 1271-1285. G e v e r s T . , G h e b r e a b S., S m e u l d e r s A. (1998), Colour Invariant Snakes, Proc. BM VC,
578-588, B M V A .
K l i n k e r G. , S h a f e r A. , K a n a d a T. (1990), A P hysical Approach to Colour Im age Understanding, “International Journal o f Computer V ision” , 4, 7-38.
P a u w l e s E., F r e d e r i x G . (1999), N on-P aram etric Clustering fo r Im age S egm entation and G rouping, “C om puter Vision and Image U nderstanding” , 75, nos. 1/2, 73-85.
S k a r b e k W., K o s c h a n A. (1994), Colour Im age Segm entation: A. Survery, “Technischer Bericht”, 32, Technical University o f Berlin.
J e r z y K o r z e n ie w s k i
M E T O D Y SE G M E N T A C JI K O LO R Ó W W O B R A Z A C H D W U W Y M IA R O W Y C H
Artykuł składa się z dw óch części. Pierwsza zawiera przegląd wybranych m etod segmentacji kolorów w obrazach dw uwym iarow ych. W drugiej części zaproponow any jest now y algorytm segmentacji kolorów . N ow y algorytm różni się od innych tym, że kładzie nacisk na dokładne zdefiniowanie klas k olorów oraz na jak najmniejszą ich liczbę (warunkowo względem ustalonych wartości param etrów), natom iast mniejszy nacisk jest położon y na otrzym anie małej liczby segmentów. D ziałanie algorytm u jest ocenione poprzez porów nanie segmentacji typow ego obrazu dw uw ym iarow ego z segmentacjami w ykonanym i przez inne algorytm y.
