: Neural network weights typically follow a normal distribution. NF4 concentrates its 16 "bins" where most weights exist (near zero), minimizing rounding errors.
: A process that quantizes the quantization constants themselves to save additional memory. NF4.rar
: A feature to handle memory spikes during training by offloading to CPU RAM. 🔬 Key Technical Details : Neural network weights typically follow a normal
💡 : If you are looking for the software/machine learning paper, search for "QLoRA" or "4-bit NormalFloat" on arXiv . : A feature to handle memory spikes during
The paper explains why NF4 is superior to standard 4-bit integers (Int4) or floating-point (Float4) formats:
: Compresses 16-bit weights to 4 bits, effectively reducing VRAM usage by ~75% (e.g., a 65B parameter model can be loaded with ~35GB instead of ~130GB).
: RNF4 mediates the degradation of the PML-RARα fusion protein.
