Vitaliy Vashchenko Vitaliy Vashchenko - 2 months ago 21
Swift Question

NSLayoutManager hides new line characters no matter what I do

I'm trying to show invisible characters like the new line character in my NSTextView subclass. The usual approach like overriding drawGlyph method of NSLayoutManager is a bad idea because it's too slow and not work properly with multi-paged layout.

What I'm trying to do is to override the setGlyph method of the NSLayoutManager so it would replace invisible "\n" glyph with "¶" glyph and " " with "∙".

And it works on the " " space glyphs but has no effect on the new line characters.

public override func setGlyphs(_ glyphs: UnsafePointer<CGGlyph>, properties props: UnsafePointer<NSGlyphProperty>, characterIndexes charIndexes: UnsafePointer<Int>, font aFont: Font, forGlyphRange glyphRange: NSRange) {
var substring = (self.currentTextStorage.string as NSString).substring(with: glyphRange)

// replace invisible characters with visible
if PreferencesManager.shared.shouldShowInvisibles == true {
substring = substring.replacingOccurrences(of: " ", with: "\u{00B7}")
substring = substring.replacingOccurrences(of: "\n", with: "u{00B6}")
}

// create a CFString
let stringRef = substring as CFString
let count = CFStringGetLength(stringRef)

// convert processed string to the C-pointer
let cfRange = CFRangeMake(0, count)
let fontRef = CTFontCreateWithName(aFont.fontName as CFString?, aFont.pointSize, nil)
let characters = UnsafeMutablePointer<UniChar>.allocate(capacity: MemoryLayout<UniChar>.size * count)
CFStringGetCharacters(stringRef, cfRange, characters)

// get glyphs for the pointer of characters
let glyphsRef = UnsafeMutablePointer<CGGlyph>.allocate(capacity: MemoryLayout<CGGlyph>.size * count)
CTFontGetGlyphsForCharacters(fontRef, characters, glyphsRef, count)

// set those glyphs
super.setGlyphs(glyphsRef, properties:props, characterIndexes: charIndexes, font: aFont, forGlyphRange: glyphRange)
}


Then I came up with an idea: it looks like NSTypesetter marks new line char ranges like those it shouldn't process at all. So I subclassed NSTypesetter and did override a method:

override func setNotShownAttribute(_ flag: Bool, forGlyphRange glyphRange: NSRange) {
let theFlag = PreferencesManager.shared.shouldShowInvisibles == true ? false : true
super.setNotShownAttribute(theFlag, forGlyphRange: glyphRange)
}


But it's not working. NSLayoutManager still won't generate a glyph for the new line character, no matter what glyph I create.

What am I doing wrong?

Answer

As I figured out, the default implementation of NSTypesetter's setNotShownAttribute: of the class doesn't change already generated glyphs in its glyph storage. So, call of super doesn't produce any effect. I just have to replace glyphs manually before calling super.

So, the most efficient implementation of showing invisible characters (you will see the difference while zooming the view) is this:

  1. Subclass NSLayoutManager and override setGlyphs to show space chars:

    public override func setGlyphs(_ glyphs: UnsafePointer<CGGlyph>, properties props: UnsafePointer<NSGlyphProperty>, characterIndexes charIndexes: UnsafePointer<Int>, font aFont: Font, forGlyphRange glyphRange: NSRange) {
        var substring = (self.currentTextStorage.string as NSString).substring(with: glyphRange)
    
        // replace invisible characters with visible
        if PreferencesManager.shared.shouldShowInvisibles == true {
            substring = substring.replacingOccurrences(of: " ", with: "\u{00B7}")
        }
    
        // create a CFString
        let stringRef = substring as CFString
        let count = CFStringGetLength(stringRef)
    
        // convert processed string to the C-pointer
        let cfRange = CFRangeMake(0, count)
        let fontRef = CTFontCreateWithName(aFont.fontName as CFString?, aFont.pointSize, nil)
        let characters = UnsafeMutablePointer<UniChar>.allocate(capacity: MemoryLayout<UniChar>.size * count)
        CFStringGetCharacters(stringRef, cfRange, characters)
    
        // get glyphs for the pointer of characters
        let glyphsRef = UnsafeMutablePointer<CGGlyph>.allocate(capacity: MemoryLayout<CGGlyph>.size * count)
        CTFontGetGlyphsForCharacters(fontRef, characters, glyphsRef, count)
    
        // set those glyphs
        super.setGlyphs(glyphsRef, properties:props, characterIndexes: charIndexes, font: aFont, forGlyphRange: glyphRange)
    }
    
  2. Subclass NSATSTypesetter and assign it to your NSLayoutManager subclas. The subclass will display the new line characters and make sure that every invisible character will be drawn with a different color:

    class CustomTypesetter: NSATSTypesetter {
    
        override func setNotShownAttribute(_ flag: Bool, forGlyphRange glyphRange: NSRange) {
            var theFlag = flag
    
            if PreferencesManager.shared.shouldShowInvisibles == true   {
                theFlag = false
    
                // add new line glyphs into the glyph storage
                var newLineGlyph = NSGlyph(100)
                self.substituteGlyphs(in: glyphRange, withGlyphs: &newLineGlyph)
    
                // draw new line char with different color
                self.layoutManager?.addTemporaryAttribute(NSForegroundColorAttributeName, value: NSColor.invisibleTextColor, forCharacterRange: glyphRange)
            }
    
            super.setNotShownAttribute(theFlag, forGlyphRange: glyphRange)
        }
    
        /// Currently hadn't found any faster way to draw space glyphs with different color
        override func setParagraphGlyphRange(_ paragraphRange: NSRange, separatorGlyphRange paragraphSeparatorRange: NSRange) {
            super.setParagraphGlyphRange(paragraphRange, separatorGlyphRange: paragraphSeparatorRange)
    
            guard PreferencesManager.shared.shouldShowInvisibles == true else { return }
    
            if let substring = (self.layoutManager?.textStorage?.string as NSString?)?.substring(with: paragraphRange) as NSString? {
                substring.enumerateSubstrings(in: NSRange(location: 0, length: substring.length), options: .byWords) { (sub, range, enclosingRange, stop) in
    
                    // substring control character after word
                    let controlRange = NSRange(location: NSMaxRange(range), length: NSMaxRange(enclosingRange) - NSMaxRange(range))
                    let controlString = substring.substring(with: controlRange)
    
                    for charIndex in 0 ..< controlString.characters.count {
                        // if it's a space character, then draw it with different color
                        let controlSubRange = NSRange(location: charIndex, length: 1)
                        if (controlString as NSString).substring(with: controlSubRange) == " " {        
    
                            // translate local subtring range into the global scope
                            let globalSubRange = NSRange(location: paragraphRange.location + controlRange.location + controlSubRange.location, length: 1)
                            self.layoutManager?.addTemporaryAttribute(NSForegroundColorAttributeName, value: Color.invisibleTextColor, forCharacterRange: globalSubRange)
                        }
                    }
                }
            }
        }
    }
    
  3. To show/hide invisible characters just call:

    let storageRange = NSRange(location: 0, length: currentTextStorage.length)
    layoutManager.invalidateGlyphs(forCharacterRange: storageRange, changeInLength: 0, actualCharacterRange: nil)
    layoutManager.ensureGlyphs(forGlyphRange: storageRange)
    
Comments