Mapped, but no dice

Finished mapping what I think are all of the pointing sources. However, the RMS of the centroids has not gone down. Now I need to go through each individual source and image and try to get rid of the baddies. There are presumably some sources with bulk offsets that can be removed to improve the fits. I just have to find them.

Nothing

No progress today. Spent the whole day re-mapping pointing sources and figuring out which observations ARE pointing sources. Mapping STILL not done - it takes a long time when there are hundreds of separate observations. Also, finding some corrupt files and noting them in /scratch/adam_work/texts/toolarge.txt . A lot more of that work to be done.

Observation distribution

-bash-3.00$ wc texts/all_*_observations.txt                                                                                                 
    0      0      0 texts/all_0505_observations.txt
    0      0      0 texts/all_0506_observations.txt
  151    151   6039 texts/all_0507_observations.txt
   33     33   1377 texts/all_0509_observations.txt
   16     16    633 texts/all_0605_observations.txt
  135    135   5309 texts/all_0606_observations.txt
    0      0      0 texts/all_0607_observations.txt
   94     94   3780 texts/all_0609_observations.txt
   34     34   1386 texts/all_0705_observations.txt
   26     26   1028 texts/all_0706_observations.txt
  201    201   7989 texts/all_0707_observations.txt
   73     73   2907 texts/all_0709_observations.txt
    0      0      0 texts/all_polychrome_0505_observations.txt
  222    222  11898 texts/all_polychrome_0506_observations.txt
  471    471  24861 texts/all_polychrome_0507_observations.txt
  196    196  10346 texts/all_polychrome_0509_observations.txt
   43     43   2200 texts/all_polychrome_0605_observations.txt
  730    730  38040 texts/all_polychrome_0606_observations.txt
   22     22   1195 texts/all_polychrome_0607_observations.txt
  471    471  24751 texts/all_polychrome_0609_observations.txt
   66     66   3399 texts/all_polychrome_0705_observations.txt
  145    145   7533 texts/all_polychrome_0706_observations.txt
 1051   1051  54801 texts/all_polychrome_0707_observations.txt
  511    511  27049 texts/all_polychrome_0709_observations.txt
 4691   4691 236521 total

L111 observation notes

In L111, the failures noted in the previous post are specific to September 2005. By comparing the pointing model images to the "raw cso pointing" and "no pointing model" (the latter still have FAZO/FZAO applied, all that has been done to them is aberration/nutation and precession correction) it appears that the Sep 05 model is simply wrong.

I haven't had time to check on W3/4/5 and L033, but presumably they have a similar issue. W3/4/5 includes late July 07 and September 06. L033 includes July 05, June 06, and July 07. I will need to generate pointing plots for September 05 and 06, and probably regenerate June 06. I have the rest and I think Meredith's models are fine for those dates - they certainly don't create recognizable errors in the maps.

It is possible that my pointing_model.pro has some pointing models (the problematic ones) entered incorrectly. It would be nice to get those double-checked.

Pointing model failure

I attempted to map L111, L033, and W3/4/5 with the pointing model corrections applied. That was a failure. Two images are attached to illustrate the problem - relative pointing is clearly not correct (middle of L33, bottom left of L111).

So.... possibilities:

1. Pointing models are incorrect
2. Pointing model signs have been misinterpreted (don't think this one is possible based on other graphs)
3. Uhh... ideas?

Next step, I'll use the individual maps with no ptg mdl and find out what the offsets should be....

News, modifications

RA / Dec mapping did change centroid locations, but not really for the better. A deeper analysis is probably necessary, but nevertheless I'm not convinced mapping type is the problem. The list of possible answers to the question of why my pointing models don't match Meredith's:

1. The new mapping is shifting the centroid
   
2. The centroiding method I'm using is different / incorrect
   
3. There is still something in the pointing we haven't caught
   

The lack of systematics suggests that the third option is not correct. The first seems the most likely, but also the most difficult to track down. I don't know where to go with #2.


I've updated the code so that the pointing model correction offsets in RA/Dec are written to the FITS header. This will be the standard in all future runs for individual observations. The calculation of this offset post-mapping is straightforward but it's nice to have a redundant error check.


The report for Monday will be kind of empty, sadly. However, I think I can say that I'll now move on to testing the previous problems we faced, in particular that we could not match the pointing across whole fields (L111). If THAT is fixed, then at least we know that whatever my pipeline is doing differently (e.g. PA at all times...) is useful if not 100% correct.

I'll need to be in the office to test the field mappings, though, because ds9 display doesn't work well over wireless. I'll do a remote run of the individual L111 and W5 maps to compare to previous ones.

Pointing update

I believe I've completely dealt with the projection issues. There are a lot of new plots available now, pretty much everything for Jun/Jul 05 and July 07. Low on my to-do list is the rest of the pointing model tests.

My conclusion now is that I can't match the 3-4" RMS in dAlt and dAz. Even with aggressive sigma-rejection of outliers, I only get down to ~10" RMS (which adds to ~15" RMS). My calculated pointing models differ significantly from Meredith's, but not extremely so for 0707.

The code I've used to generate the plots is centroid_plots.pro under plotting/. Using that code, I am able to essentially reproduce each change made in do_the_pointing besides the distortion mapping. Because of the weak dependence of the pointing model on altitude, even errors on the scale of precession won't really hurt the pointing model calculations. The only unanswered question I think may be responsible for some errors - but not enough, I think - is whether eq2hor/hor2eq could be double-correcting for aberration and nutation, but this wouldn't affect the actual data reduction. It WOULD affect my plotting!

I think my next test is to try to coadd different epochal data for particular fields, e.g. l001, l033. Not l000 because of the galactic coordinate issue.