Linux ocr for getting text from a screenshot

These OCR programs are probably not calibrated for making text out of pixel-perfect low-resolution screen shots, but from high-resolution somewhat noisy scans of different type faces on paper. Doing OCR from a screenshot ought to be quite easy: Each letter is pixel perfect and looks exactly the same, and there are no problems with slanting text or other distortions. In fact, writing your own OCR program is a distinct possibility for this.

I had via mail received a 72dpi screenshot that I wanted to get the text from. The top part looked like this:

The top of the screenshot

Tesseract, which is a program that is highly recommended on the web, returned nothing when run on this screenshot. At first after reading this, I thought this had to do with my tif possibly having a layer of transparency, but ensuring it was not there did not change anything.

According to the same discussion, it seems like tesseract wants to have a high resolution image (see tests on that further down).

Now the ocrad program returned this:

Al Po_ Po_ _ Al _|_o _|_o _ M_|_o_hl_o
lollob_IOldo _|_o_do l_m_o
A_ _ol__lo _ _|_o_do l_m_o
__ _o _| Colmo_ _ ____ _ ___ T__o_
OIOo Ml__ __o C_o_o_o_ O_O____o
llo_o_do__ _ l_|_ __llO_ Co__ol__
_| Mo_|___o O_O____o A_oOo_
A_ Amo_ C_o_do Woblo_ lo_ Ml_odo_ _ Alb___o Bo__o_ _| Tl_o_ D
_o_ldo B___lol _ _|_hl_ _o_ b Bobb_ c___
lo_ Tomoll_o_ D_ OIOo O_O____o A_oOo_ _o__
A_ld _o_ Bo_____o
_o_ Wo_ Wo_ B___o _|__o C_bo_ Plo____

 ...and so on.

Gocr returned this:

y 9 999)        9       9     _J   ypp yyy    y
AI P a n P a n Y AI Vin o Vin o             m eIc o c hita
L oIIo brigid a                    Ric ard o L e m v o
A V aIeria                    Ric ard o L e m v o
Se Va EICaiman                 fru ko Y Sus Tesos
Oiga mire Vea                   Guayacan Orquesta
LIora n d ote                    L uis F eIi e G o n z aIe z
EImanisero                    Orquesta Aragȯn
A Amor Cuando HabIan Las miradas       AIberto Barros''EITitan D.
S o nid o B e stiaI                  Ric hie R a 6 B o b b C ru z
Los TamaIitos De OIga               Orquesta Aragȯn,Josė
A cid                       R a y B arretto
Ran Kan Kan                   Buena Vista Cuban PIayers
Undanta                     Bo Kas ers Orkester

...and so on, which given the non-outputting competition, must be deemed fantastic. Still, it cannot deal with any characters extending below the baseline (p, g and y for example), and all ls are interpreted as 1s.

Increasing the pixel density of the image

This turned out to be non trivial with the tools I had at hand. I finally got resampling working with the program pnmenlarge, part of the netpbm suite of command line unixish image processing tools:

cat salsatext.pnm | pnmenlarge 4 > enlarged.pnm

This quadrupled each pixel, and now tesseract magically started working!

 (convert to to tif first)

Fil Pen Pen 'ii Fil '·.·'inp '·.·'inp et`} i···1el¤:p·:hite
Lpllplznrigide Riterdp Lem·-rp
.·!·.·-,# '·.·'elerie et`} Riterdp Lem·-rp
5e '·.·'e El Ceimen et`} Frulce 'ii 5us Tesps
Ciige i···1ire '·.·'ee Gueyeten Ordueste
Llprendpte et`} Luis Felipe Gpneelee
El i···1eniserp Ordueste ifiregdn
.·!·.·-,# Famer, Cuendp Hel:·len Les i···1iredes et`} .·!·.ll:¤ertp Eerrps "El Titen D. ..
Epnidp Eestiel et`} Richie Re·-,# El Epl:·l:·3r Crue
Lps Temelitps De Cilge Ordueste ifiregdn, _|pse  
.·!·.¤:id Re·-,# Eerrettp
Ren Ken Ken Euene '·.·'iste ·Zul:·en F‘le3··ers
Llndenteg et`} Ep iiespers Orltester

Well, it does at least produce output, but the quality is at the point that you can barely guess which line it is trying to decode.

Let's try switching to Spanish as language:

.ü.| Pan Pan "x‛ .ü.| '·.·'ina '·.·'ina —l=*.`Š f'·'1a|·:·:··:|'•i|:a
La||a|:·ri·;|i·:|a F‘xi·:ar·:|a Lam'­.«a
.ü.'-,« '·.·'a|aria —l=*.`Š F‘xi·:ar·:|a Lam'­.«a
Sa '·.·'a El Caiman —l=*.`Š FrukJ:· "x‛ 5uS TaSaS
Diga Mira '·.·'aa Cuaşracan Dr·:]uaS|:a
Llarandata —l=*.`Š LuiS Falipa Ganzalaz
El f'·'1aniSara Dr·:]uaS|:a Aragón
.ü.'-,« Fumar, Cuanda Ha|:·|an LaS f'·'1ira·:|aS —l=*.`Š .ü.||:·ar|:·:· EarraS "EI Titan D. ..
Sanida EaS|:ia| —l=*.`Š F‘xi·:|'•ia Ra'-; Ex Ea|:·|:·ş» Cruz
LaS Tama|i|:aS Da Diga Dr·:]uaS|:a Aragón, _|aSa  
.ü.·:i·:| Ra'-; Earratta
F‘xan Kan Kan Euana '·.·'iSta Cu|:·an F‘|aş·'arS
L|n·:|an|:a·; —l=*.`Š Ba kaS|:·arS DrkaStar

That was not good. Maybe the enlargement needs to be smoother?

pamstretch, also from the netpbm package, also increases pixel count but additionally smooths the output by interpolating pixels.

As many unixish tools, pamstretch takes data from stdin and outputs it to stdout:

cat salsatext.pnm | pamstretch 4 > stretched.pnm

Tesseract needs tif format, handled here by Imagemagick's convert command

convert  stretched.pnm  stretched.tif

run tesseract on it in this case with -l spa, which means Spanish language

tesseract stretched.tif str -l spa

The result:

AI Pan Pan 'l" AI Vino Vino —.?•.`$ Molcochita
Lollobrigida Ricardo Lomyo
Ay 'o‘aloria —.?•.`$ Ricardo Lomyo
5o 'o‘a El Caiman —.?•.`$ Fruko 'l" Sus Tosos
Diga Miro 'o‘oa Guayacan ûrquosta
Llorandoto —.?•.`$ Luis Folipo Gonzalo:
El Manisoro ûrquosta Aragon
Ay Amor, Cuando Hablan Las Miradas —.?•.`$ Alborto Barros "El Titan D. ..
5onido Eostial —.?•.`$ Richio Ray En Bobby Cruz
Los Tamalitos Do Olga ûrquosta Aragon, José  
Acid Ray Earrotto
Ran Kan Kan Euona 'liista Cuban Playors
Undantag —.?•.`$ Bo Iäaspors ûrkostor

...better. Let's try English:

AI Pan Pan 'i" AI 'a'inu 'a'inu 3} Ms|cuchita
Lu||ubrigic|a Ricarclu Lsmyu
Ay 'a'a|sria 3} Ricarclu Lsmyu
5s 'a'a El Caiman 3} Fruku 'i" 5us Tssus
Diga Mirs 'a'sa Guayacan Drqussta
L|uranduts 3} Luis Fs|ips Gun:a|s:
El Manissru Drqussta Aragun
Ay Amur, Cuanclu Hab|an Las Miradas 3} Albsrtu Earrus "El Titan D. ..
5unic|u Esstia| 3} Richis Ray E: Eubby Cru:
Lus Tama|itus Ds D|ga Drqussta Aragun, juss  
Acid Ray Earrsttu
Ran Iian Iian Eusna 'a'ista Cuban Playsrs
Unclantag 3} Eu Iiaspsrs Drksstsr

That is worse.

How does ocrad perform?

Al Pan Pan Y Al Vino Vino __ Melcochila
Lollobrigida Ricardo Lemvo
Ay Valeria __ Ricardo Lemvo
Se Va El Caiman __ FrukD Y Sus Tesos
Oiga Mire Vea Guayacan Orquesla
Llorandole __ Luis Felipe Gonzalez
El Manisero Orquesla Arag�n
Ay Amor, Cuando Nablan Las Miradas __ Alberlo Barros "El Tilan D,,,
Sonido Beslial __ Richie Ray bBobby Cruz
Los Tamalilos De Olga Orquesla Arag�n, los� , , ,
Acid Ray Barrello
Ran Kan Kan Buena Visla Cuban Players
Undanlag __ Bo Kaspers OrkPsler

A lot better than the line noise seen before. With enlarged but not interpolated:

Al Pan Pan Y Al Vino Vino __ Melcochi_a
Lollobrigida Ricardo Lemvo
Ay Valeria __ Ricardo Lemvo
Se Va El Caiman __ FrukoYSusTesos
Oiga Mire Vea Cuayacan Orques_a
Llorando_e __ Luis Felipe Conzalez
El Manisero Orques_a Arag�n
Ay Amor, Cuando Hablan Las Miradas __ Alber_o Barros "El Ti_an D,,,
Sonido Bes_ial __ Richie Ray b Bobby Cruz
Los Tamali_os De Olga Orques_a Arag�n, _os� ,,,
Acid Ray Barre__o
Ran Kan Kan Buena Vis_a Cuban Players

That's worse.

So, tesseract and ocrad needs the input to be "scannified" by multiplying the pixel count and interpolating to get a bit of smoothness, but they still do not clearly beat gocr.

For scanned in documents the ranking seems reversed.

 Peter Selinger: Review of Linux OCR software:

Of course, it must be stressed that the test results reported here are derived from only two scanned pages. It is possible that for other inputs, the programs rank differently. However, based on the tests reported on this page, here is a summary of my conclusions:
* Tesseract gives extremely good output at a reasonable speed. It is the clear overall winner of the test. The only caveat is that one absolutely must convert the input to bitonal.
* Ocrad gives reasonable output at extremely high speed. It can be useful in applications where speed is more important than accuracy.
* GOCR gives poor output at a slow speed.