Karl Norris - Fond memories, deep gratitude and future need

Contribution from Harald Martens

 Bio-chemometrician,

harald.martens@idletechs.com

Karl Norris, kind giant Karl Norris helped us to make sense out of real-world data. He focused on multi-channel data from diffuse high-speed spectrophotometers, primarily in the near-infrared (NIR) wavelength range.

Learning from Karl Karl helped and inspired me in several ways, over many years. With my background in psychometrics and chemometrics, I was familiar with multivariate data analysis. But as food biochemist I did not have measurements to match it. 

          Around 1980 my friend and mentor in Copenhagen, Lars Munck, sent me to visit Karl in his USDA lab in College Park, Maryland. At that time I had already worked desperately for 8 years to develop methods and tools to extract information from slow chromatographs, to find variation patterns hidden behind lots of rather noisy chemical peaks. Frankly, I felt a bit lonely. But when Karl showed me around in his lab, I felt at home immediately. I later visited him several times, and he introduced me to many of his friends in the wonderful NIR community.

Karl and I never discussed politics and religion, - we would probably have disagreed about some things, for we came from different cultures. But I think we shared a common view of how to understand the physical world and about how we as researchers ought to use our time here. We clearly agreed about the importance of measuring real-world qualities in real-world samples to solve real-world problems. Gradually, I learnt to see the value of math, statistics, computers and programming, and of having the human in the loop when combining our prior knowledge and our modern measurements. Karl showed me his interactive version of “machine learning” - how he developed calibration models for getting value out of per se meaningless NIR spectra from his scanning CARY instrument. Mathematically, his system was based on multiple regression on statistical combinations of sums and ratios of measurements at different wavelength channels. Statistically, it was very parsimonious and thus easy to validate. But it was also graphic and highly interactive, - it allowed him to use his human background knowledge. Different from anything I had seen in statistical software, including my own! Using spectral derivatives in chosen wavelength regions, his interactive regression plots revealed and quantified additive absorbance variation patterns - allowing him to correct for the unwanted variations, known or unknown.

OK, through the ‘70s I had also discovered some of that, working long nights alone at the University’s main computer to make psychometric PCA and MLR useful for chemists. But by interactively ratioing pairs of spectral derivatives, he also removed unwanted, multiplicative path length effects in the same process! On top of that, he interactively eliminated wavelength regions that he did not find useful. I returned home, surprised, confused and deeply impressed.

The MSC was just an attempt to copy Karl.  At the Norwegian Food Research Institute we got our first NIR reflectance instrument (Infralyzer 400) in 1981. The instrument had only 19 fixed wavelength filters, but it was easy to use, and gave very, very precise reflectance data R. One day, after I had tried - in vain- to mimic Karl’s spectral derivative ratioing for our fixed-filter instrument, I plotted each food sample’s “19-channel absorbance spectrum” on the y-axis against their average “absorbance spectrum” on the x-axis. Immediately, a statistical equivalent of Karl’s derivative ratioing became obvious: The different samples’ spectra formed different, almost straight lines. Subtracting for the line offsets mimicked Karl’s spectral derivatives and, at the same time, dividing by the line slopes, mimicked his multiplicative spectral derivative ratioing. I named the method “Multiplicative Scatter Correction, MSC”.  Later, after we saw that the same preprocessing was also applicable to chromatographic profiles and taste panel evaluation data, I started to call it “Multiplicative Signal Correction”. In the future I believe we shall see a merger of modelling principles from the MSC-type preprocessing and PCA/PLSR-type subspace data modelling, in a dynamic feedback setting of control theory. I am still surprised at how useful is mathematics.

But for me personally, it has all been strongly based on Karl’s warm inspiration and courageous ideas, and on my jealousy of his Cary scanning spectrophotometer.

Harald