v2.0 is to v1.0 as ___ is to v1.0

Apparently v2.0 recovers flux better at all scales than v1.0.  However, it is noisier.  The full comparison is available here.  The trustworthy range of flux scales is 1.2-1.6ish.  The flux is increased at ALL scales in most images.  However, as was noted in comparisons with MAMBO, there may be curvature in the best-fit lines.  Also, l=35 does not appear to scale up linearly... G34 is fainter in v2.0?

In the low S/N fields, the PSD is more reliable than the pixel-pixel comparison.

PPS comparison completed

The PPS scale factor seems to hold for many different (and ridiculous) aperture sizes.  The full results are here.

PPS analysis: suggests a possible solution to the discrepancy

The comparisons I mentioned in the previous post are sort of done.  They are pretty suggestive of a solution to the "flux recovery problem" we think must be true.  However, even if it is a solution, it doesn't really solve the problem completely.

It looks like v1.0 should be scaled up by a factor of 1.3-1.4 (not 1.5).  v2.0 is consistent with the PPS sources to within 5%, and might even be slightly too high.

The comparison was done by taking the flux in a 60" radius aperture (equivalent to bolocat 120" diameter apertures) and subtracting off the background measured in a 120" radius annulus around the source.  Without the background subtraction, these numbers would look very different: in the science fields, most of the sources sit on an extended background.  Even though the "background flux" isn't recovered in the PPS fields, it should contribute to the source background because it is involved in the atmosphere subtraction (it's sort of "already subtracted" so you have to subtract from the science fields).

Next step: direct comparison between v1.0 and v2.0.  Pixel by pixel, aperture, and powerspectrum

PPS analysis

I've started looking at PPS fields to see if I can glean any additional information about the "flux discrepancy" from them.  However, the results are, as usual, unenlightening.

There is no consistent increase in flux when 3 PCA components are used instead of 13 PCA components - very plausibly an indication that 13 PCA is not too much to subtract because it's only atmosphere.  Similarly, there is no obvious benefit to using a quadratic sky estimator instead of a PCA estimator.

I'm using aperture photometry (without background subtraction) on identical fields to perform these comparisons.  I've selected (arbitrarily) the l357pps source as my comparison source.  The next step (ongoing) is to compare to the co-added maps and crosshatched large-scale maps of the same field.

(next step) PPS < single cross-hatched large-scale observation pair < 13PCA full combined map < 3PCA full combined map.

Unfortunately, this result implies that the small maps under-recover flux, which suggests that the large maps are too bright, which is the opposite of what we expect.  Additionally, lower noise -> more flux recovered?

When background subtraction is included, the 3PCA and 13PCA fluxes match nearly perfectly.

Despite the failure of this test (PPS < full field), I will do a systematic comparison of PPS sources with 0PCA + masking to the large fields in the hopes that doing so can provide a legitimate estimate of the "scale factor" from treating small and large fields differently.

BGPS Pipeline Published

The BGPS pipeline has officially been released on Google Code.

Aperture Photometry on Herschel-based simulation

Aperture Photometry on isolated and not-so-isolated sources in the Herschel-based BGPS simulation using the L=111 field for the "noise". Depending on the aperture, our flux recovery can be really really low. The images should give an idea of the S/N.  Background subtraction means subtracting the median of the image.... it works frighteningly well in most cases.

Pipeline Flowcharts

In the process of hunting down a supposed calibration error, I determined that it was necessary to generate a more intuitive graphical display of the pipeline. Hence, pipeline flowcharts (generated in keynote). The key should be self-explanatory to the degree that any part of these charts is understandable to an outsider. The yellow boxes represent wrapper scripts/functions while the rounded box bubbles show individual functions within these wrappers and their interrelationship.