Saturday, November 8, 2008

Latest re-run of the pipeline

Wow. A lot of errors this time around.


bash: /scratch/adam_work/l020/l020_fitslist.txt: Permission denied
MRDFITS: Image array (1634,560) Type=Real*4
% HEULER: FITS header is already in Galactic: nothing changed
% READFITS: ERROR - Unable to locate file
060628_o32_raw_ds5.nc_indiv13pca_map01.fits
% Variable is undefined: HDIMAGE.
% Execution halted at: IMAGE_SHIFTS 44
/home/milkyway/student/ginsbura/bgps_pipeline/alignment/image_shifts.pro
% COALIGN_FIELD 31
/home/milkyway/student/ginsbura/bgps_pipeline/wrappers/coalign_field.pro
% $MAIN$


Updated radec_offsets in /scratch/sliced_polychrome/l034/050708_o21_raw_ds5.nc
Writing offsets to -0.00192975 -9.22609e-09 (ncfile was -0.000337092)
% ALIGNMENT_OFFSETS_TO_NCDF: Could not find file
% Execution halted at: ALIGNMENT_OFFSETS_TO_NCDF 4
/home/milkyway/student/ginsbura/bgps_pipeline/preproc/alignment_offsets_to_
ncdf.pro
% WRITE_IMSHIFTS 11
/home/milkyway/student/ginsbura/bgps_pipeline/alignment/write_imshifts.pro
% COALIGN_FIELD 34
/home/milkyway/student/ginsbura/bgps_pipeline/wrappers/coalign_field.pro
% IMAGE_SHIFTS 44
/home/milkyway/student/ginsbura/bgps_pipeline/alignment/image_shifts.pro
% COALIGN_FIELD 31
/home/milkyway/student/ginsbura/bgps_pipeline/wrappers/coalign_field.pro
% $MAIN$


/scratch/adam_work/l354/070709_o16_raw_ds5.nc_indiv13pca took 110.42429 sec.

MRDFITS: File access error
% HEULER: ERROR - First parameter must be a FITS header or astrometry
structure
% Execution halted at: HEULER 78

FIELD l351 COMPLETED at Sat Nov 8 05:34:01 2008. Took 1122.3564 seconds
% Program caused arithmetic error: Floating underflow
% Program caused arithmetic error: Floating illegal operand
FIELD l052 BEGUN at Sat Nov 8 05:34:01 2008
% FILE_LINES: Error opening file.
File: /scratch/sliced/l052/l052_infile.txt
No such file or directory
% Error occurred at: READCOL 141

FIELD l359 COMPLETED at Sat Nov 8 05:41:34 2008. Took 452.60051 seconds
% Program caused arithmetic error: Floating underflow
% Program caused arithmetic error: Floating illegal operand
FIELD l066 BEGUN at Sat Nov 8 05:41:34 2008
% FILE_LINES: Error opening file.
File: /scratch/sliced/l066/l066_infile.txt
No such file or directory
% Error occurred at: READCOL 141


FIELD l072 BEGUN at Sat Nov 8 05:41:34 2008
% FILE_LINES: Error opening file.
File: /scratch/sliced/l072/l072_infile.txt
No such file or directory
% Error occurred at: READCOL 141



MRDFITS: Image array (628,561) Type=Real*4
% HEULER: FITS header is already in Galactic: nothing changed
% HEULER: FITS header is already in Galactic: nothing changed
/scratch/adam_work/l086/060627_o23_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -9.77712 11.8034 shifts in arcsec l/b:
70.395277 84.984804 delta-ra (as): 19.007974
delta-dec (as): -108.70337 x/y error in pixels: 3.99631
0.0701244 ra/dec error: -0.0079926204 0.00014024887 gal l/b error:
-0.0079926204 0.00014024887
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 203.757 -0.224758
so a different method is being attempted
WARNING! x,y offs were too large on the SECOND TRY too: -74.7339
-159.448 so a third method is being attempted
This third method is meant to deal with small overlap regions that may be confused by high noise, it restricts
to a maximum 15 pixel offset (really less than that in practice) and therefore should be checked carefully.
/scratch/adam_work/l086/060627_o26_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -1.38579 1.39157 shifts in arcsec l/b:
9.9777001 10.019338 delta-ra (as): 1.1521393
delta-dec (as): -14.093081 x/y error in pixels: 0.0814682
0.0882818 ra/dec error: -0.00016293635 0.00017656364 gal l/b error:
-0.00016293635 0.00017656364
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 9.13210 11.3905
so a different method is being attempted
WARNING! x,y offs were too large on the SECOND TRY too: -79.0260
20.0907 so a third method is being attempted
This third method is meant to deal with small overlap regions that may be confused by high noise, it restricts
to a maximum 15 pixel offset (really less than that in practice) and therefore should be checked carefully.
/scratch/adam_work/l086/070707_o49_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -9.13210 -11.3905 shifts in arcsec l/b:
65.751123 -82.011258 delta-ra (as): -105.06683
delta-dec (as): 3.1919479 x/y error in pixels: 0.486307
0.950778 ra/dec error: -0.00097261386 0.0019015568 gal l/b error:
-0.00097261386 0.0019015568
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 0.384630 13.1890
so a different method is being attempted
WARNING! x,y offs were too large on the SECOND TRY too: 205.434
-23.9600 so a third method is being attempted
This third method is meant to deal with small overlap regions that may be confused by high noise, it restricts
to a maximum 15 pixel offset (really less than that in practice) and therefore should be checked carefully.
/scratch/adam_work/l086/070708_o37_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -0.384630 -13.1890 shifts in arcsec l/b:
2.7693373 -94.960636 delta-ra (as): -74.063617
delta-dec (as): 59.509407 x/y error in pixels: 2.29135
0.933383 ra/dec error: -0.0045826990 0.0018667649 gal l/b error:
-0.0045826990 0.0018667649
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: -143.139 10.8130
so a different method is being attempted
% Subscript range values of the form low:high must be >= 0, < size, with low
<= high: CORR.
% Execution halted at: PIXSHIFT 107




ALL PREPROC ... output is /scratch/adam_work/l133/070911_o25_raw_ds5.nc_indiv13pca READING IN 1 FILES FROM /scratch/sliced/l133/070911_o25_raw_ds5.nc
Reading file /scratch/sliced/l133/070911_o25_raw_ds5.nc
Applying pointing model with alt/az off -0.024687220 -0.029154389
NO OFFSETS APPLIED
3.6277909 sec.
No values flagged as bad!
AC_BOLOS has 0.0 NAN points, which is 0.0 of total
Flatfielding (polynomial subtraction) ...0.92621708 sec.
No values flagged as bad!
AC_BOLOS has 0.0 NAN points, which is 0.0 of total
Prepare Map ... 1.4727209 sec.
% CHECK_FITS: NAXIS keywords in FITS header have been updated
Done preproc ... 11.113711 sec.
ITERATION NUMBER 0 ...SKY SUBTRACTION - MEDIAN0.15759706 sec.
BEGINNING PCA SUBTRACTION OF COMPONENTS 0 TO 13 ... 0.68130398 sec.
Flatfielding (polynomial subtraction) ...0.96450186 sec.
% CHECK_FITS: NAXIS keywords in FITS header have been updated
% CHECK_FITS: NAXIS keywords in FITS header have been updated
% Program caused arithmetic error: Floating underflow
Deconvolution iteration 0: 1.2357290 sec.
Deconvolution iteration 1: 0.93198013 sec.
Deconvolution iteration 2: 0.93152189 sec.
Deconvolution iteration 3: 0.93164992 sec.
Chi2 for iteration 0 is 1.22593e+07 with 2.07110e+06 degrees of freedom
FINISHED ITERATION 0 13.803532 sec.
ITERATION NUMBER 1 ...SKY SUBTRACTION - MEDIAN0.15691710 sec.
BEGINNING PCA SUBTRACTION OF COMPONENTS 0 TO 13 ... % TRIQL: Too many iterations in tqli.
% Error occurred at: EIGENQL 122
/usr/local/itt/idl70/lib/eigenql.pro
% PCA_SUBTRACT 19



/scratch/adam_work/l135p1/060917_o12_raw_ds5.nc_indiv13pca took 30.478553 sec.

MRDFITS: File access error
% HEULER: ERROR - First parameter must be a FITS header or astrometry
structure
% Execution halted at: HEULER 78


MRDFITS: Image array (1630,555) Type=Real*4
% HEULER: FITS header is already in Galactic: nothing changed
% HEULER: FITS header is already in Galactic: nothing changed
/scratch/adam_work/l068/070910_ob3_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -0.0271645 -0.0270424 shifts in arcsec l/b:
0.19558409 -0.19470519 delta-ra (as): -0.26866187
delta-dec (as): -0.062403024 x/y error in pixels: 0.000193238
0.00126581 ra/dec error: -3.8647498e-07 2.5316247e-06 gal l/b error:
-3.8647498e-07 2.5316247e-06
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 34.8510 -5.72119
so a different method is being attempted
WARNING! x,y offs were too large on the SECOND TRY too: 142.704
76.4267 so a third method is being attempted
This third method is meant to deal with small overlap regions that may be confused by high noise, it restricts
to a maximum 15 pixel offset (really less than that in practice) and therefore should be checked carefully.
/scratch/adam_work/l068/070911_ob8_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: 15.0101 3.42143 shifts in arcsec l/b:
-108.07276 24.634271 delta-ra (as): 78.221561
delta-dec (as): 78.545365 x/y error in pixels: 0.0114306
0.184410 ra/dec error: -2.2861189e-05 0.00036881918 gal l/b error:
-2.2861189e-05 0.00036881918
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 56.2296 14.6994
so a different method is being attempted
WARNING! x,y offs were too large on the SECOND TRY too: 728.753
-184.087 so a third method is being attempted
This third method is meant to deal with small overlap regions that may be confused by high noise, it restricts
to a maximum 15 pixel offset (really less than that in practice) and therefore should be checked carefully.
/scratch/adam_work/l068/070911_ob9_raw_ds5.nc_indiv13pca_map01.fits
shifts in pixels: -56.2296 -14.6994 shifts in arcsec l/b:
404.85292 -105.83592 delta-ra (as): -304.51436
delta-dec (as): -286.88182 x/y error in pixels: 7.32951
0.00354093 ra/dec error: -0.014659024 7.0818565e-06 gal l/b error:
-0.014659024 7.0818565e-06
% HEULER: FITS header is already in Galactic: nothing changed
WARNING! x,y offs were too large: 3.31224 -13.8321
so a different method is being attempted
% Subscript range values of the form low:high must be >= 0, < size, with low
<= high: CORR.
% Execution halted at: PIXSHIFT 107

/scratch/adam_work/l111/060605_o48_raw_ds5.nc_indiv13pca took 10.098406 sec.

MRDFITS: File access error
% HEULER: ERROR - First parameter must be a FITS header or astrometry
structure
% Execution halted at: HEULER 78


Writing offsets to /scratch/sliced/l027/060616_o10_raw_ds5.nc 0.00120866
-0.00287235 (ncfile was 060616_o10_raw_ds5.nc)
Updated radec_offsets in /scratch/sliced/l027/060616_o10_raw_ds5.nc
Writing offsets to 5.77312e-05 -2.39760e-06 (ncfile was 0.000409983)
% ALIGNMENT_OFFSETS_TO_NCDF: Could not find file
% Execution halted at: ALIGNMENT_OFFSETS_TO_NCDF 4

7 comments:

Adam said...

somehow /scratch/sliced_polychrome/l034/050708_o21_raw_mmd_ds5.nc got into the l033_infile and screwed things up

Adam said...

/scratch/sliced/l354/070724_o10_raw_ds5.nc was the reference file for l354 but somehow was excluded from the l354_infile. Warrants another check post-mapping

Adam said...

I think l086 needs to be re-treated specially; it had way too many large offsets. Probably the error that resulted was from an overlap region smaller than itself....

Adam said...

curiously, l135p1 has a reference field that is only present in the w3ohbolocam directory. Probably need to remake the infiles for all of W3/4/5

Adam said...

the l111 reference field was 60% flagged out because of bad rotators:

ALL PREPROC ... output is /scratch/adam_work/l111/060602_o34_raw_ds5.nc_indiv13pca READING IN 1 FILES FROM /scratch/sliced/l111/060602_o34_raw_ds5.nc
Reading file /scratch/sliced/l111/060602_o34_raw_ds5.nc
Flagged scan 2 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -23.2941
Flagged scan 6 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -24.9265
Flagged scan 7 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -21.3750
Flagged scan 8 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -25.7647
Flagged scan 9 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.110294
Flagged scan 10 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -26.6140
Flagged scan 11 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.143383
Flagged scan 12 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -27.4743
Flagged scan 14 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -28.3235
Flagged scan 15 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -24.6397
Flagged scan 17 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -25.4559
Flagged scan 18 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -30.0441
Flagged scan 19 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.110292
Flagged scan 21 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.220589
Flagged scan 22 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -31.8088
Flagged scan 23 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -27.9596
Flagged scan 24 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -32.7022
Flagged scan 26 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -33.5846
Flagged scan 28 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -34.4890
Flagged scan 30 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -35.3934
Flagged scan 31 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -31.3787
Flagged scan 33 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -0.231617
Flagged scan 34 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.121325
Flagged scan 36 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.132353
Flagged scan 38 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.209558
Flagged scan 39 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by -0.198528
Flagged scan 40 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.231617
Flagged scan 42 in file /scratch/sliced/l111/060602_o34_raw_ds5.nc
because rotangle changes by 0.121325
Did not find rpcfile /data/bgps/raw/rpc/20060602/20060602_0811_rpc.bin for file /scratch/sliced/l111/060602_o34_raw_ds5.nc
Applying pointing model with alt/az off -0.00023445157 -0.00016187002
NO OFFSETS APPLIED
Scans 2 6 7 8 9 10
11 12 14 15 17 18
19 21 22 23 24 26
28 30 31 33 34 36
38 39 40 42
flagged in file /scratch/sliced/l111/060602_o34_raw_ds5.nc because of rotation
4.2049680 sec.
Total of 1947176 points flagged
AC_BOLOS has 1.94718e+06 NAN points, which is 0.608696 of total

Adam said...

redo targets:
l068
l086
l111
l027

Adam said...

l020 too