Unit Converter: From Floating-Point Precision to Temperature Offsets#

I recently needed to convert between inches and centimeters for a project. Wrote a quick formula, but the result didn’t match the design specs—that’s when I realized unit conversion isn’t as simple as it looks.

Basic Conversion: The Base Unit Method#

The obvious approach is direct formulas:

// Inch to centimeter
const inchToCm = (inch) => inch * 2.54
// Centimeter to inch
const cmToInch = (cm) => cm / 2.54

Problem: with 10 length units, you’d need 90 conversion functions. Not practical.

A better approach is the base unit method: pick one unit as the base (e.g., meter), and store only the conversion factor relative to that base.

interface Unit {
  name: string
  symbol: string
  factor: number  // Multiplier relative to base unit
}

const lengthUnits = {
  m:  { name: 'Meter', symbol: 'm', factor: 1 },        // Base unit
  km: { name: 'Kilometer', symbol: 'km', factor: 1000 },
  cm: { name: 'Centimeter', symbol: 'cm', factor: 0.01 },
  mm: { name: 'Millimeter', symbol: 'mm', factor: 0.001 },
  in: { name: 'Inch', symbol: 'in', factor: 0.0254 },
  ft: { name: 'Foot', symbol: 'ft', factor: 0.3048 },
  mi: { name: 'Mile', symbol: 'mi', factor: 1609.344 },
}

Convert by going through the base unit:

function convert(value: number, from: Unit, to: Unit): number {
  const baseValue = value * from.factor  // Convert to base unit (meters)
  return baseValue / to.factor           // Convert to target unit
}

// 10 miles = ? kilometers
convert(10, lengthUnits.mi, lengthUnits.km)  // 16.09344

Now N units only need N factor definitions—much easier to maintain.

The Temperature Exception#

Temperature can’t use simple factor conversion because their zero points differ. 0°C is water’s freezing point, 0°F is a brine mixture temperature, and 0K is absolute zero.

Temperature conversion needs an offset:

const tempUnits = {
  c: { name: 'Celsius', symbol: '°C', factor: 1, offset: 0 },
  f: { name: 'Fahrenheit', symbol: '°F', factor: 5/9, offset: -32 },
  k: { name: 'Kelvin', symbol: 'K', factor: 1, offset: -273.15 },
}

The conversion logic changes:

function convertTemperature(value: number, from: Unit, to: Unit): number {
  // First convert to Celsius (as base)
  let celsius = (value + from.offset) * from.factor

  // Then convert to target unit
  return celsius / to.factor - to.offset
}

// 100°C = ? °F
convertTemperature(100, tempUnits.c, tempUnits.f)  // 212

A detail here: factor: 5/9, not factor: 9/5. Converting Fahrenheit to Celsius uses (F - 32) × 5/9, so factor is 5/9.

Floating-Point Precision Issues#

Unit converters often hit precision problems:

0.1 + 0.2  // 0.30000000000000004
0.1 * 3    // 0.30000000000000004

This causes weird results in conversions:

// 1/3 foot to inches
const feet = 1/3
const inches = feet * 12  // 3.9999999999999996

Solutions:

1. Reasonable Decimal Places# 

function formatResult(value: number): string {
  // Keep at most 6 decimals, strip trailing zeros
  return value.toFixed(6).replace(/\.?0+$/, '')
}

formatResult(3.9999999999999996)  // "4"

2. Use a Precision Library#

For exact calculations (finance, science), use decimal.js:

import Decimal from 'decimal.js'

function convertPrecise(value: number, from: Unit, to: Unit): string {
  const baseValue = new Decimal(value).times(from.factor)
  return baseValue.div(to.factor).toDecimalPlaces(6).toString()
}

3. Choose Appropriate Factor Precision#

Factor precision affects final results. A mile is exactly 1609.344 meters, not 1609:

// Imprecise
mi: { factor: 1609 }

// Precise
mi: { factor: 1609.344 }

Errors compound: 100 miles × 0.344 m/mi = 34.4 meters of error.

Binary Prefix vs Decimal Prefix#

Data storage has a gotcha: is KB 1000 bytes or 1024 bytes?

// Option 1: Decimal prefix (hard drive manufacturers)
const dataUnits = {
  KB: { factor: 1000 },
  MB: { factor: 1000000 },
  GB: { factor: 1000000000 },
}

// Option 2: Binary prefix (operating systems)
const dataUnits = {
  KB: { factor: 1024 },
  MB: { factor: 1048576 },
  GB: { factor: 1073741824 },
}

That’s why your 500GB hard drive shows only 465GB on your computer: manufacturers use 1000, systems use 1024.

500 * 1000 * 1000 * 1000 / (1024 * 1024 * 1024)  // 465.66 GB

In a unit converter, you need to specify which standard you use. I chose binary (1024) since developers are more familiar with it.

Performance: Avoid Recalculation#

When users frequently switch units, cache intermediate results:

function useConverter(category: string, value: number) {
  const [baseValue, setBaseValue] = useState(0)

  // Update base value when input changes
  useEffect(() => {
    const from = getCurrentUnit(category)
    setBaseValue(value * from.factor)
  }, [value, category])

  // Use cached base value when switching units
  const convertTo = useCallback((to: Unit) => {
    return baseValue / to.factor
  }, [baseValue])

  return { convertTo }
}

The Result#

Based on these ideas, I built: Unit Converter

Features:

  • 6 categories: length, weight, area, volume, temperature, data storage
  • Accurate floating-point calculations
  • Real-time bidirectional conversion

The core code is only about 200 lines, but handling precision, temperature offsets, and binary prefixes properly makes it reliable.

Related tools: PX REM Converter | Timestamp Converter