summaryrefslogtreecommitdiff
path: root/src/hb-ot-shape-complex-indic-machine.rl
AgeCommit message (Collapse)AuthorFilesLines
2018-10-02[indic/use] Factor common expressions in ragel machineBehdad Esfahbod1-4/+6
No machine change.
2018-10-02[indic] Minor flip grammar aroundBehdad Esfahbod1-1/+1
No behavior change.
2018-10-02[ragel] Use ts/te (token-start / token-end) instead of manual trackingBehdad Esfahbod1-5/+3
2018-10-01[indic] Remove Consonant_Medial from grammar and codeBehdad Esfahbod1-6/+4
The only Indic CM is U+0A75 GURMUKHI SIGN YAKASH, which Uniscribe doesn't treat specially, and font designers design for that. So, do the same.
2018-09-10[indic] Do NOT allow matra after Halant,ZWJBehdad Esfahbod1-1/+1
Fixes https://github.com/harfbuzz/harfbuzz/issues/556 Devanagari regresses 12 tests, and Gujarati 2. See: https://github.com/harfbuzz/harfbuzz/issues/556#issuecomment-419957472 New numbers: BENGALI: 353725 out of 354188 tests passed. 463 failed (0.130722%) DEVANAGARI: 707299 out of 707394 tests passed. 95 failed (0.0134296%) GUJARATI: 366353 out of 366457 tests passed. 104 failed (0.0283799%) GURMUKHI: 60729 out of 60747 tests passed. 18 failed (0.0296311%) KANNADA: 951300 out of 951913 tests passed. 613 failed (0.0643966%) MALAYALAM: 1048136 out of 1048334 tests passed. 198 failed (0.0188871%) ORIYA: 42327 out of 42329 tests passed. 2 failed (0.00472489%) SINHALA: 271596 out of 271847 tests passed. 251 failed (0.0923313%) TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%) TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
2018-08-25Rename hb-*private.hh to hb-*.hhBehdad Esfahbod1-1/+1
Sorry for the noise, downstream custom builders. Please adjust.
2018-04-24Minor, re-add unused mark to some of ragel defined state machines (#1005)Ebrahim Byagowi1-1/+1
2018-04-23Resolve clang's used-but-marked-unused warningsEbrahim Byagowi1-1/+1
2018-01-05MinorBehdad Esfahbod1-1/+0
2018-01-05[indic] Remove unused OT_VD categoryBehdad Esfahbod1-2/+1
2018-01-05MinorBehdad Esfahbod1-1/+0
2018-01-05[indic] Remove more Khmer-specific stuff from Indic shaperBehdad Esfahbod1-5/+3
No numbers changed.
2017-10-03[indic] Implement Consonent_With_StackerBehdad Esfahbod1-2/+3
Fixes https://github.com/behdad/harfbuzz/issues/528
2016-05-06[indic] Remove unused Javanese bitsBehdad Esfahbod1-2/+1
2014-12-10[indic] Allow ZWJ/ZWNJ before SMBehdad Esfahbod1-1/+1
In Oriya, a ZWJ/ZWNJ might be added before candrabindu to encourage or stop ligation of the candrabindu. This is clearly specified in the Unicode section on Oriya. Allow it there. Note that Uniscribe doesn't allow this. Micro tests added using Noto Sans Oriya draft. No changes in numbers. Currently at: BENGALI: 353725 out of 354188 tests passed. 463 failed (0.130722%) DEVANAGARI: 707307 out of 707394 tests passed. 87 failed (0.0122987%) GUJARATI: 366349 out of 366457 tests passed. 108 failed (0.0294714%) GURMUKHI: 60732 out of 60747 tests passed. 15 failed (0.0246926%) KANNADA: 951190 out of 951913 tests passed. 723 failed (0.0759523%) KHMER: 299070 out of 299124 tests passed. 54 failed (0.0180527%) MALAYALAM: 1048147 out of 1048334 tests passed. 187 failed (0.0178378%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271662 out of 271847 tests passed. 185 failed (0.068053%) TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%) TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
2014-05-29[indic] Do NOT allow reph formation on placeholdersBehdad Esfahbod1-1/+1
Only allow it on DOTTED CIRCLE. No effect on test numbers. Test: U+0930,094D,00A0
2014-05-27[indic/myanmar] Rename OT_NBSP to OT_PLACEHOLDERBehdad Esfahbod1-2/+2
2014-05-23[indic] Simplify grammarBehdad Esfahbod1-4/+3
No functional change.
2014-05-23[indic] Always start new syllable for AvagrahaBehdad Esfahbod1-1/+1
In fact, the previous grammar was ambigious. No functional change.
2014-05-23[indic] Rename avagraha cluster to symbol clusterBehdad Esfahbod1-5/+5
In anticipation of adding more characters to that class of clusters.
2014-05-22[indic] Recategorize U+A8E0..A8F1 as OT_VDBehdad Esfahbod1-1/+1
Up to two of them come after all OT_A characters.
2014-05-22[indic] Allow up to three tone marksBehdad Esfahbod1-1/+1
According to Roozbeh, there are valid combinations in Unicode proposals for up to three. Previously we were allowing up to two.
2013-10-24[indic] Fix CM2, reallyBehdad Esfahbod1-1/+1
Followup from 6e613f3365bf4e9fd778758c53e7de00c64beca1.
2013-10-18[indic] Add Javanese support!Behdad Esfahbod1-6/+8
Seems to be working just fine!
2013-10-16[indic] Improve Avagraha support in machineBehdad Esfahbod1-2/+4
2013-10-16[indic] Misc harmless fixes!Behdad Esfahbod1-4/+4
First, we were abusing OT_VD instead of OT_A. Fix that but moving OT_A in the grammar where it belongs (which is different from what the spec says). Also, only allow medial consonants after all other consonants. This doesn't affect any current character. Finally, fix Halant attachment in presence of medial consonants. Again, this currently doesn't affect any sequence. I lied. There's Gurmukhi U+0A75 which is Consonant_Medial. Uniscribe allows one of those in each of these positions: before matras, after matras and before syllable modifiers, and after syllable modifiers! We currently just allow unlimited numbers of it, before matras.
2013-10-16[indic] Allow up to two syllable modifiersBehdad Esfahbod1-1/+1
Bug 70509 - Candrabindu+Visarga doesn't work in Devanagari https://bugs.freedesktop.org/show_bug.cgi?id=70509 We categorize both bindus and visarga as syllable-modifiers. OT spec doesn't actually say what characters go in the syllable modifier category, and allows one. We just allow up to two now. Test case: U+0930,U+0941,U+0901,U+0903 Uniscribe currently doesn't support that and produces a dotted circle.
2013-10-15[indic] Handle AvagrahaBehdad Esfahbod1-1/+4
It can come either at the end(ish!) of the syllable, or independently. When independent, it accepts a few bits and pieces.
2013-02-14Fix compile warnings for ragel-generated machinesBehdad Esfahbod1-1/+1
2012-12-21[Indic] Insert dottedcircle after a lone Malayalam dot-rephBehdad Esfahbod1-1/+1
2012-12-05Fix warningsBehdad Esfahbod1-1/+1
2012-11-12[Indic] Allow Consonant_Medial's after Consonant'sBehdad Esfahbod1-1/+2
Mostly affects Myanmar, but also Tai Tham, Javanese, and Cham. The latter three are untested (no fonts!).
2012-09-07[Indic] Find syllables before any features are appliedBehdad Esfahbod1-2/+2
With FreeSerif, it seems that the 'ccmp' feature does ligature substituttions. That was then causing syllable match failures. We now find syllables before any features have been applied. Test sequence: U+0D9A,U+0DCA,U+200D,U+0DBB,U+0DCF
2012-09-05[Indic] Allow "H,ZWJ,M"Behdad Esfahbod1-1/+1
Uniscribe accepts a Halant,ZWJ before matras. Allow that. BENGALI down from 295 to 291 DEVANAGARI down from 69 to 57 GUJARATI down from 19 to 17 KANNADA down from 871 to 867 MALAYALAM down from 340 to 337 TELUGU down from 20 to 16 Currently at: BENGALI: 353897 out of 354188 tests passed. 291 failed (0.0821598%) DEVANAGARI: 707337 out of 707394 tests passed. 57 failed (0.00805774%) GUJARATI: 366440 out of 366457 tests passed. 17 failed (0.00463902%) GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%) KANNADA: 951046 out of 951913 tests passed. 867 failed (0.0910798%) KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%) LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%) MALAYALAM: 1047997 out of 1048334 tests passed. 337 failed (0.0321462%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%) TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%) TELUGU: 970557 out of 970573 tests passed. 16 failed (0.00164851%) TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-09-05[Indic] Relax grammarBehdad Esfahbod1-2/+2
Now that we insert dotted-circle, tests break more easily when our indic machine breaks. In particular, a few Devanagari tests were having sequences like "C,H,ZWJ,N", and because of the ZWJ the Nukta does NOT get reordered to before the Halant as the grammar used to expect... Fixup. Another case is as simple as "C,ZWJ,SM". Fixes 10 out of 79 failures: DEVANAGARI: 707325 out of 707394 tests passed. 69 failed (0.00975411%)
2012-09-05[Indic] Fix Khmer syllable-final coeng-consonantBehdad Esfahbod1-1/+1
Brings down Khmer failures from 162 to 47. KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%) Also rebaselined some of the test files that had only-inherited lines. Removing those, the stats are: BENGALI: 353893 out of 354188 tests passed. 295 failed (0.0832891%) DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%) GUJARATI: 366438 out of 366457 tests passed. 19 failed (0.00518478%) GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%) KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%) KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%) LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%) MALAYALAM: 1047994 out of 1048334 tests passed. 340 failed (0.0324324%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%) TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%) TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%) TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%) Still some regressions, but some of the more egregious cases are addressed.
2012-08-31[Indic] Implement dotted-circle insertion for broken clustersBehdad Esfahbod1-1/+3
No panic, we reeally insert dotted circle when it's absolutely broken. Fixes most of the dotted-circle cases against Uniscribe. (for Devanagari fixes 80% of them, for Khmer 70%; the rest look like Uniscribe being really bogus...) I had to make a decision. Apparently Uniscribe adds one dotted circle to each broken character. I tried that, but that goes wrong easily with split matras. So I made it add only one dotted circle to an entire broken syllable tail. As in: "if there was a dotted circle here, this would have formed a correct cluster." That works better for split stuff, and I like it more.
2012-08-31[Indic] Start adding dotted-circle instrastructureBehdad Esfahbod1-10/+9
2012-08-02[Indic] Move more stuff to the shape_planBehdad Esfahbod1-2/+2
Almost done. Need to add per-script static tables.
2012-08-02[OT] Pipe shape_plan down to pause_callbacksBehdad Esfahbod1-2/+2
2012-07-24[Indic] Limit matras to 4 per syllableBehdad Esfahbod1-2/+2
Also limit joiners. This limits our syllable length to a constant, and is closer to what Uniscribe does anyway. Two Devanagari tests regressed, but who cares about tests with 20 joiners in a row?! Devanagari at 57 (0.00821766%) now.
2012-07-24[Indic] End Vowel-based syllable at ZWJBehdad Esfahbod1-1/+1
One Devanagari test regressed, plus 10 Malayalam (at 1545 now). Fixed 120 Sinhala failures. Now at 208 (0.0765136%).
2012-07-23[Indic] Limit syllables to at most five consonantsBehdad Esfahbod1-3/+3
Seems to be about what Uniscribe does. Not exactly. But close enough. More consonants will start a new cluster. A few scripts went way down in failures. In particular: - Devanagari failures went down from 490 to 56. - Telugu went down from 113 to 49. Other scripts went down slightly or didn't change. New numbers: BENGALI: 353908 out of 354285 tests passed. 377 failed (0.106412%) DEVANAGARI: 693572 out of 693628 tests passed. 56 failed (0.00807349%) GUJARATI: 366485 out of 366506 tests passed. 21 failed (0.00572978%) GURMUKHI: 60750 out of 60809 tests passed. 59 failed (0.0970251%) KANNADA: 950730 out of 951913 tests passed. 1183 failed (0.124276%) KHMER: 298613 out of 299124 tests passed. 511 failed (0.170832%) MALAYALAM: 1046881 out of 1048416 tests passed. 1535 failed (0.146411%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271333 out of 271847 tests passed. 514 failed (0.189077%) TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%) TELUGU: 970524 out of 970573 tests passed. 49 failed (0.00504856%) Some of the remaining Telugu and Devanagari issues seem to be Uniscribe eating Anusvara when placed before a non-joiner. Ouch!
2012-07-20[Indic] Allow a ZWNJ after SM'sBehdad Esfahbod1-1/+1
Malayalam failures go way down. Other scripts benefitted slightly too. Sinhala had one or two test regressions, but...
2012-07-20[Indic] Break syllables at Halant,ZWNJBehdad Esfahbod1-2/+3
That's really what Uniscribe does, and explains a lot of pecularities of Halant,ZWNJ before the base. Sent Telugu from 1% failures to 0.03%. Improved Kannada and Malayalam slightly. Fixed half of Bengali, and did NOT break anything!
2012-07-18[Indic] Accept a forced Rakar sequence at the end of syllableBehdad Esfahbod1-1/+2
In Sinhala, Rakar is formed by Al-Lakuna,ZWJ,Ra. If you put that at the end of a Consonant,Matra syllable, you get a dotted-circle from Uniscribe. Apparently adding a ZWJ before the Al-Lakuna "fixes" that. And people have been encoding that sequence... So, allow a forced "ZWJ,Virama,ZWJ,Ra" sequence at the of syllables. Fixes some 100 or more of Sinhala failures. Now at 622 only (0.23%).
2012-07-18[Indic] Allow joiners on both sides of Halant at the same timeBehdad Esfahbod1-1/+1
The sequence <ZWJ,Al-Lakuna,ZWJ> is used in Sinhala to explicitly ask for Rakar. Fixes two-thousand Sinhala tests. Not many left.
2012-07-18[Indic] Treat Register Shifters like NuktaBehdad Esfahbod1-1/+1
Really this time. Fixes another 18 Khmer tests.
2012-07-18[Indic] Allow joiners before matrasBehdad Esfahbod1-2/+2
Fixes 1 more Devanagari test!
2012-07-18[Indic] Allow halant group in Vowel and placeholder syllablesBehdad Esfahbod1-3/+4
Fixes 2 out of 560 Devanagari failures. AND: Fixes 1 out of 2 Tamil failures.
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-18 10:29:45 +0000