Deep Dive into Linux less Command: Building a Web Pager from Scratch

Last week, I was debugging a server issue and needed to check a massive log file. cat flooded my terminal, vim was overkill, but less saved the day. This seemingly simple command packs some brilliant engineering.

Core Design Philosophy#

The name less is a pun on more — “less is more”. Its killer feature is forward navigation: you can start viewing without loading the entire file.

Traditional more could only scroll forward and required the entire file in memory. less broke these limitations:

Streaming reads: Only load what’s visible
Bidirectional navigation: Scroll forward, backward, search and jump
Real-time tracking: Monitor file changes like tail -f

Memory Management: Handling Huge Files#

Two key data structures in less source code:

// Simplified implementation
struct filestate {
    off_t file_pos;        // Current file position
    off_t screen_start;    // Screen start line
    struct line *lines;    // Line buffer (fixed size)
    int line_count;        // Parsed line count
};

struct line {
    char *text;            // Line content
    off_t file_offset;     // Offset in file
};

The trick is lazy parsing: only parse lines around the current viewport, pre-parse ahead when scrolling forward, cache displayed lines when scrolling back.

In a web environment, we can simulate this:

class VirtualScroller {
  private buffer: Map<number, string> = new Map()
  private bufferSize = 100 // Cache 50 lines above and below

  async getLine(lineNumber: number): Promise<string> {
    // Cache hit
    if (this.buffer.has(lineNumber)) {
      return this.buffer.get(lineNumber)!
    }

    // Calculate range to load
    const startLine = Math.max(0, lineNumber - this.bufferSize / 2)
    const lines = await this.loadLines(startLine, this.bufferSize)

    // Update cache (LRU policy)
    lines.forEach((line, i) => {
      this.buffer.set(startLine + i, line)
    })

    return lines[lineNumber - startLine]
  }

  private async loadLines(start: number, count: number): Promise<string[]> {
    const chunk = await this.readFileChunk(start, count)
    return chunk.split('\n')
  }
}

Search Algorithm: Efficient Regex Implementation#

less supports regex search (/pattern for forward, ?pattern for backward). In JavaScript:

class SearchEngine {
  private regex: RegExp | null = null
  private matches: number[] = [] // Matched line numbers

  search(pattern: string, lines: string[], direction: 'forward' | 'backward') {
    try {
      this.regex = new RegExp(pattern, 'gi')
      this.matches = []

      lines.forEach((line, i) => {
        if (this.regex!.test(line)) {
          this.matches.push(i)
        }
      })

      return direction === 'forward'
        ? this.matches[0]
        : this.matches[this.matches.length - 1]
    } catch (e) {
      console.error('Invalid regex:', e)
      return -1
    }
  }

  findNext(currentLine: number, direction: 'forward' | 'backward') {
    if (!this.matches.length) return -1

    if (direction === 'forward') {
      return this.matches.find(n => n > currentLine) ?? this.matches[0]
    } else {
      return [...this.matches].reverse().find(n => n < currentLine)
        ?? this.matches[this.matches.length - 1]
    }
  }
}

Performance tip: Incremental search. As the user types each character, only search visible lines. Search the entire file only after they press Enter.

Real-time Tracking: +F Flag Implementation#

less +F log.txt monitors file changes like tail -f. The implementation:

class FileWatcher {
  private lastSize = 0
  private fileHandle: FileHandle | null = null

  async watch(filePath: string, onAppend: (newContent: string) => void) {
    this.fileHandle = await fs.open(filePath, 'r')
    this.lastSize = (await this.fileHandle.stat()).size

    // File change detection (Node.js)
    fs.watch(filePath, async (eventType) => {
      if (eventType === 'change') {
        const newSize = (await this.fileHandle!.stat()).size
        if (newSize > this.lastSize) {
          const buffer = Buffer.alloc(newSize - this.lastSize)
          await this.fileHandle!.read(buffer, 0, buffer.length, this.lastSize)
          onAppend(buffer.toString())
          this.lastSize = newSize
        }
      }
    })
  }
}

In the browser, we can use FileReader + polling:

class WebFileWatcher {
  async watch(file: File, onAppend: (content: string) => void) {
    let lastSize = 0

    setInterval(async () => {
      if (file.size > lastSize) {
        const chunk = file.slice(lastSize)
        const text = await chunk.text()
        onAppend(text)
        lastSize = file.size
      }
    }, 1000) // Check every second
  }
}

less shortcuts are heavily influenced by Vim:

Key	Function	Web Implementation
`j` / `↓`	Next line	`scrollBy(0, lineHeight)`
`k` / `↑`	Previous line	`scrollBy(0, -lineHeight)`
`Space` / `f`	Next page	`scrollBy(0, clientHeight)`
`b`	Previous page	`scrollBy(0, -clientHeight)`
`g`	Go to start	`scrollTop = 0`
`G`	Go to end	`scrollTop = scrollHeight`
`/pattern`	Search forward	Open search box
`?pattern`	Search backward	Open search (reverse flag)
`n`	Next match	`findNext()`
`N`	Previous match	`findPrev()`
`q`	Quit	Close viewer

Keyboard event handler:

function handleKeyboard(e: KeyboardEvent, viewer: FileViewer) {
  // Search mode
  if (viewer.searchMode) {
    if (e.key === 'Enter') {
      viewer.executeSearch()
      viewer.searchMode = false
    } else if (e.key === 'Escape') {
      viewer.searchMode = false
    } else {
      viewer.searchQuery += e.key
    }
    return
  }

  // Normal mode
  switch (e.key) {
    case 'j':
    case 'ArrowDown':
      viewer.scrollLine(1)
      break
    case 'k':
    case 'ArrowUp':
      viewer.scrollLine(-1)
      break
    case ' ':
    case 'f':
      viewer.scrollPage(1)
      e.preventDefault() // Prevent page scroll
      break
    case 'b':
      viewer.scrollPage(-1)
      break
    case '/':
      viewer.startSearch('forward')
      e.preventDefault()
      break
    case '?':
      viewer.startSearch('backward')
      e.preventDefault()
      break
    case 'n':
      viewer.jumpToNextMatch()
      break
    case 'N':
      viewer.jumpToPrevMatch()
      break
  }
}

Performance Boundaries & Optimizations#

Challenges with huge files (GB scale):

Line index building: Parsing all newlines is slow
- Solution: Parse in chunks, only index visited parts
Search performance: Full-text regex can freeze
- Solution: Web Worker background search with progress bar
Memory usage: Caching all lines explodes memory
- Solution: LRU cache + compressed storage (deduplicate repeated lines)

class OptimizedLineCache {
  private lineIndex: Int32Array // File offset index
  private contentCache: Map<number, string> = new Map()
  private maxCacheSize = 1000

  async getLine(lineNumber: number): Promise<string> {
    if (this.contentCache.has(lineNumber)) {
      return this.contentCache.get(lineNumber)!
    }

    // Locate and read based on index
    const start = this.lineIndex[lineNumber]
    const end = this.lineIndex[lineNumber + 1] ?? Infinity
    const buffer = await this.readFileRange(start, end)

    // LRU eviction
    if (this.contentCache.size >= this.maxCacheSize) {
      const firstKey = this.contentCache.keys().next().value
      this.contentCache.delete(firstKey)
    }

    this.contentCache.set(lineNumber, buffer)
    return buffer
  }
}

Real-world Use Cases#

Log viewing: less +F /var/log/nginx/access.log for real-time monitoring
Code review: less -N main.js with line numbers for easy reference
Config files: less /etc/nginx/nginx.conf quick view without accidental edits
Compressed files: less archive.tar.gz directly view archives (auto-invokes zless)

Summary#

The less command appears simple but embodies sophisticated engineering: streaming processing, memory management, and intuitive UX. When implementing similar web functionality, borrow these designs:

Stream reads instead of bulk loading
Cache strategy vs memory tradeoffs
Keyboard navigation conventions
Search and real-time tracking optimizations

Next time you need to view large files, try the File Viewer at JsonKit, or just type less in your terminal.

Related Tools:

Linux sed Stream Editor - Text stream processing
Linux awk Text Processing - Data extraction and reporting
Linux grep Pattern Matching - Text search utility