Urdu Unicode Explained: Why Your Text Breaks and How to Fix It

You paste Urdu into a document and it looks fine. Your colleague opens the same file and sees boxes, or letters that refuse to join, or text running the wrong way. Nothing about the file changed — so what happened?

Almost always, the answer is Unicode.

What Unicode actually is

Unicode assigns every character a permanent number. The Urdu letter ک is U+06A9, always and everywhere. When two systems agree on those numbers, text moves between them intact.

The trouble starts when a character looks like the one you want but carries a different number.

Problem 1: Arabic lookalike letters

This is by far the most common cause of broken Urdu, and the most invisible.

Urdu and Arabic share a script but not an alphabet. Several Urdu letters have Arabic counterparts that render almost identically in some fonts:

Why this is so damaging

Because many fonts draw both versions the same way, the problem stays hidden until it suddenly matters:

• Search breaks. Searching for کتاب will not find a document that used Arabic kaf. The strings are genuinely different.
• Rendering breaks. A font with Urdu glyphs but no Arabic ones shows a box, or falls back to a mismatched font mid-word.
• Sorting and comparison break. Two visually identical words compare as unequal.

How it gets in

Copying from Arabic websites, older Windows keyboard drivers, some phone keyboards, and machine translation output are the usual sources.

The fix

Run the text through a normaliser that maps the Arabic codepoints to their Urdu counterparts. Our Unicode Converter does this in Fix Unicode mode.

Problem 2: Arabic Presentation Forms

Arabic-script letters change shape depending on where they sit in a word — initial, medial, final or isolated. Normally your font handles this automatically through OpenType shaping.

But Unicode also contains a legacy block, U+FB50–U+FEFF, holding every one of those shapes as a separate character. These are the Presentation Forms, and they exist only for backwards compatibility with pre-Unicode systems.

Where they come from

Copying Urdu out of a PDF is the classic source. Many PDF generators store each glyph as its presentation form, so what you paste is a string of pre-shaped letters rather than real text.

Why it is a problem

Presentation Forms are not searchable, do not re-shape correctly when the text is edited, and often break line-wrapping. The text looks acceptable until you try to do anything with it.

The fix

Normalise the presentation-form ranges back to their base letters. Use Presentation → Unicode mode in the converter.

Problem 3: Invisible characters

Zero-width joiners, zero-width non-joiners, byte order marks and bidirectional control characters are all legitimate — Urdu genuinely needs ZWNJ in some compound words.

But they accumulate. Text copied through several apps picks up strays that cause words to break in odd places, spacing to look wrong, or comparisons to fail for no visible reason.

Stripping the ones you did not intend usually fixes mysterious spacing bugs immediately.

Problem 4: Legacy InPage and font-encoded Urdu

Before Unicode support was widespread, Urdu publishing ran on InPage with fonts like Jameel Noori Nastaleeq. These stored Urdu as ordinary Latin characters that only looked like Urdu because a specific font was installed.

Open such a file without that font and you see gibberish like myra nam where Urdu should be.

Why this still matters

An enormous amount of Urdu publishing — newspapers, books, government documents — sits in this format. Converting it to Unicode is often the only way to make decades of material searchable.

The fix and its caveat

Our converter's InPage → Unicode mode maps the common InPage encoding to real Unicode. But these encodings were never standardised across publishers, so treat the output as a strong starting point that still needs proofreading — not a guaranteed conversion.

Diagnosing your own text

If Urdu is misbehaving, work through this in order:

1. Does it search correctly? Search for a word you can see. If it is not found, you likely have Arabic lookalikes or Presentation Forms.
2. Does it look right in a different font? Switch to a known-good Nastaliq font. If it breaks, the codepoints are wrong.
3. Does it survive a round trip? Paste into a plain text editor and back. Rich-text formatting can hide problems.
4. Are letters joining? Unjoined letters usually mean Presentation Forms or a font without Urdu shaping.

Prevention

• Type Urdu with a Unicode-native tool rather than a font-encoded one
• Prefer copying from web pages over PDFs where you have the choice
• Normalise text once when it enters your workflow, not repeatedly afterwards
• When receiving Urdu from others, run it through a normaliser before adding it to a document you will search later

The short version

Urdu Unicode problems come down to four causes: Arabic letters standing in for Urdu ones, Presentation Forms from PDFs, stray invisible characters, and legacy font encodings. All four are fixable, and all four are worth fixing once rather than working around forever.

You can repair any of them with the free Urdu Unicode Converter — everything runs in your browser, so nothing you paste is uploaded.