Weighting and Scaling

The "simple" relative sensitivity calibration was causing serious problems.

The assumed model for a "gain" $G$, timestream $S$, and reference timestream $R$ is:
$S = G R$

Naively, one would assume that something like
$G = median(S/R)$
would work. However, it doesn't. The distribution of $G$ values looks like:

Note that this is true no matter what reference bolometer you use . I've monte-carlo tested this and shown that the ratio $S/R<1$ for all bolometers. This is actually a well-understood phenomenon if you look at, e.g., Wall and Jenkins section on linear least squares. If both X and Y values are gaussian-distributed quantities, the linear-least-squares solution is not ideal. Similarly, but more confusing to me, is that the min($\chi^2$) is not a correct solution either, probably because you're minimizing with respect to only one variable.

The solution is Principal Component Analysis. The most correlated component is pretty much the atmosphere (not exactly, but we'll deal with that later). The important thing is that the first row of the eigenvectors gives the relative scaling of each timestream to that most correlated component. Actually the meaning has to be more carefully defined than that, but I can't figure it out right now. However, it looks like the relative scales come out as expected if you scale to a bolometer based on the first row (column?) of eigenvectors corresponding to the most correlated component.

Using PCA and the correct scaling, the results are very pretty, i.e. perfect recovery:

So... now on to the tests of everything ever...