Coverage for /wheeldirectory/casa-6.7.0-12-py3.10.el8/lib/py/lib/python3.10/site-packages/casatasks/mstransform.py: 100%

27 statements  

« prev     ^ index     » next       coverage.py v7.6.4, created at 2024-11-01 07:19 +0000

1##################### generated by xml-casa (v2) from mstransform.xml ############### 

2##################### f896d6b577e92a6032433725e483855a ############################## 

3from __future__ import absolute_import 

4import numpy 

5from casatools.typecheck import CasaValidator as _val_ctor 

6_pc = _val_ctor( ) 

7from casatools.coercetype import coerce as _coerce 

8from casatools.errors import create_error_string 

9from .private.task_mstransform import mstransform as _mstransform_t 

10from casatasks.private.task_logging import start_log as _start_log 

11from casatasks.private.task_logging import end_log as _end_log 

12from casatasks.private.task_logging import except_log as _except_log 

13 

14class _mstransform: 

15 """ 

16 mstransform ---- Split the MS, combine/separate/regrid spws and do channel and time averaging 

17 

18  

19  

20 The task mstransform can do the same functionalities available 

21 in cvel, partition, hanningsmooth and split without the need to read and write 

22 the output to disk multiple times. The main features of this task 

23 are: 

24  

25 * take an input MS or Multi-MS (MMS) 

26 * ability to create an output MS or MMS 

27 * spw combination and separation 

28 * channel averaging taking flags and weights into account 

29 * time averaging taking flags and weights into account 

30 * reference frame transformation 

31 * Hanning smoothing 

32  

33 All these transformations will be applied on the fly without any writing to 

34 disk to optimize I/O. The user can ask to create a Multi-MS in parallel using CASA's 

35 cluster infrastructure using the parameter createmms. See MPIInterface 

36 for more information on the cluster infrastructure. 

37  

38 This task is implemented in a modular way to preserve the functionalities 

39 available in the replaced tasks. One can choose which functionality to apply 

40 or apply all of them by setting the corresponding parameters to True. Note 

41 that there is an order in which the transformations are applied to the data that 

42 makes logical sense on the point of view of the data analysis. 

43  

44 This task can create a multi-MS as the output. General selection 

45 parameters are included, and one or all of the various data columns 

46 (DATA, LAG_DATA and/or FLOAT_DATA, and possibly MODEL_DATA and/or 

47 CORRECTED_DATA) can be selected. It can also be used to create a normal 

48 MS, split-based on the given data selection parameters. 

49  

50 The mstransform task creates a Multi-MS in parallel, using the CASA MPI framework. 

51 The user should start CASA as follows in order to run it in parallel. 

52  

53 1) Start CASA on a single node with 8 engines. The first engine will be used as the 

54 MPIClient, where the user will see the CASA prompt. All other engines will be used 

55 as MPIServers and will process the data in parallel. 

56 mpicasa -n 8 casa --nogui --log2term 

57 mstransform(.....) 

58  

59 2) Running on a group of nodes in a cluster. 

60 mpicasa -hostfile user_hostfile casa .... 

61 mstransform(.....) 

62  

63 where user_hostfile contains the names of the nodes and the number of engines to use 

64 in each one of them. Example: 

65 pc001234a, slots=5 

66 pc001234b, slots=4 

67  

68 If CASA is started without mpicasa, it is still possible to create an MMS, but 

69 the processing will be done in sequential. 

70  

71 The resulting WEIGHT_SPECTRUM produced by mstransform is in the statistical 

72 sense correct for the simple cases of channel average and time average, but not for 

73 the general re-gridding case, in which the error propagation formulas applicable for 

74 WEIGHT_SPECTRUM are yet to be defined. Currently, as in cvel and in the imager, 

75 WEIGHT_SPECTRUM is transformed in the same way as the other data columns. 

76 Notice that this is not formally correct from the statistical point of view, 

77 but is a good approximation at this stage. 

78  

79 NOTE: the input/output in mstransform have a one-to-one relation. 

80 input MS -- output MS 

81 input MMS -- output MMS 

82  

83 unless the user sets the parameter createmms to True to create the following: 

84 input MS -- output MMS 

85 

86 --------- parameter descriptions --------------------------------------------- 

87 

88 vis Name of input Measurement set or Multi-MS. 

89 outputvis Name of output Measurement Set or Multi-MS. 

90 createmms Create a multi-MS output from an input MS. 

91 separationaxis Axis to do parallelization across(scan,spw,auto,baseline). 

92 numsubms The number of Sub-MSs to create (auto or any number) 

93 tileshape List with 1 or 3 elements giving the tile shape of the disk data columns. 

94 field Select field using ID(s) or name(s). 

95 spw Select spectral window/channels. 

96 scan Select data by scan numbers. 

97 antenna Select data based on antenna/baseline. 

98 correlation Correlation: '' ==> all, correlation="XX,YY". 

99 timerange Select data by time range. 

100 intent Select data by scan intent. 

101 array Select (sub)array(s) by array ID number. 

102 uvrange Select data by baseline length. 

103 observation Select by observation ID(s). 

104 feed Multi-feed numbers: Not yet implemented. 

105 datacolumn Which data column(s) to process. 

106 realmodelcol Make real a virtual MODEL column. 

107 keepflags Keep *completely flagged rows* or drop them from the output. 

108 usewtspectrum Force creation of the columns WEIGHT_SPECTRUM and SIGMA_SPECTRUM in the output MS, even if not present in the input MS. 

109 combinespws Combine the input spws into a new output spw. Only supported when the number of channels is the same for all the spws. 

110 chanaverage Average data in channels. 

111 chanbin Width (bin) of input channels to average to form an output channel. 

112 hanning Hanning smooth data to remove Gibbs ringing. 

113 regridms Transform channel labels and visibilities to a different spectral reference frame. Notice that u,v,w data is not transformed. 

114 mode Regridding mode (channel/velocity/frequency/channel_b). 

115 nchan Number of channels in the output spw (-1=all). Used for regridding, together with \'start\' and \'width\'. 

116 start Start of the output visibilities. Used for regridding, together with \'width\' and \'nchan\'. It can be in different units, depending on the regridding mode: first input channel (mode=\'channel\'), first velocity (mode=\'velocity\'), or first frequency (mode=\'frequency\'). Example values: \'5\', \'0.0km/s\', \'1.4GHz\', for channel, velocity, and frequency modes, respectively. 

117 width Channel width of the output visibilities. Used for regridding, together with \'start\', and \'nchan\'. It can be in different units, depending on the regridding mode: number of input channels (mode=\'channel\'), velocity (mode=\'velocity\'), or frequency (mode=\'frequency\'. Example values: \'2\', \'1.0km/s\', \'1.0kHz\', for channel, velocity, and frequency modes, respectively. 

118 nspw Number of output spws to create in output MS. 

119 interpolation Spectral interpolation method. 

120 phasecenter Phase center direction to be used for the spectral coordinate transformation: direction measure or field index 

121 restfreq Rest frequency to use for output. 

122 outframe Output reference frame (''=keep input frame). 

123 veltype Velocity definition. 

124 preaverage Pre-average channels before regridding, when the ratio between the output and and input widths is greater than 2. 

125 timeaverage Average data in time. 

126 timebin Bin width for time averaging. 

127 timespan Span the timebin across scan, state or both. 

128 maxuvwdistance Maximum separation of start-to-end baselines that can be included in an average. (meters) 

129 docallib Enable on-the-fly (OTF) calibration as in task applycal 

130 callib Path to calibration library file 

131 douvcontsub Enable continuum subtraction as in task uvcontsub. Note: this parameter is deprecated and will be removed in an upcoming release. The task uvcontsub can be used as an alternative. 

132 fitspw Spectral window:channel selection for fitting the continuum 

133 fitorder Polynomial order for the fits 

134 want_cont Produce continuum estimate instead of continuum subtracted data 

135 denoising_lib Use new denoising library (based on GSL) instead of casacore fitting routines 

136 nthreads Number of OMP threads to use (currently maximum limited by number of polarizations) 

137 niter Number of iterations for re-weighted linear fit 

138 disableparallel Hidden parameter for internal use only. Do not change it! 

139 ddistart Hidden parameter for internal use only. Do not change it! 

140 taql Table query for nested selections 

141 monolithic_processing Hidden parameter for internal use only. Do not change it! 

142 reindex Hidden parameter for use in the pipeline context only 

143 

144 --------- examples ----------------------------------------------------------- 

145 

146  

147  

148 Detailed description of keyword arguments: 

149  

150 --- Input/Output parameters --- 

151 vis -- Name of input visibility file 

152 default: ''; example: vis='ngc5921.ms' 

153  

154 outputvis -- Name of output visibility file or Multi-MS 

155 default: ''; example: outputvis='ngc5921.mms' 

156  

157 createmms -- Create an output Multi-MS from an input MS. 

158 default: False 

159  

160 This parameter only has effect if set to True, when it will try 

161 to create an output Multi-MS from an input MS. The one-to-one 

162 relation of input/output in mstransform is: 

163 input MS -- output MS 

164 input MMS -- output MMS 

165  

166 by setting createmms=True, the following is possible: 

167 input MS -- output MMS 

168  

169 NOTE: See information on processing input Multi-MS at the end of this help section. 

170  

171 separationaxis -- Axis to do parallelization across. 

172 default: 'auto' 

173 options: 'scan', 'spw', 'auto', 'baseline' 

174  

175 * The 'auto' option will partition per scan/spw to obtain optimal load balancing with the 

176 following criteria: 

177  

178 1 - Maximize the scan/spw/field distribution across sub-MSs 

179 2 - Generate sub-MSs with similar size 

180  

181 * The 'scan' or 'spw' axes will partition the MS into scan or spw. The individual sub-MSs may 

182 not be balanced with respect to the number of rows. 

183  

184 * The 'baseline' axis is mostly useful for Single-Dish data. This axis will partition the MS 

185 based on the available baselines. If the user wants only auto-correlations, use the 

186 antenna selection such as antenna='*&&&' together with this separation axis. Note that in 

187 if numsubms='auto', partition will try to create as many subMSs as the number of available 

188 servers in the cluster. If the user wants to have one subMS for each baseline, set the numsubms 

189 parameter to a number higher than the number of baselines to achieve this. 

190  

191 numsubms -- The number of sub-MSs to create. 

192 default: 'auto' 

193 Options: any integer number (example: numsubms=4) 

194  

195 The default 'auto' is to partition using the number of available servers in the cluster. 

196 If the task is unable to determine the number of running servers, or the user did not start CASA 

197 using mpicasa, numsubms will use 8 as the default. 

198  

199 tileshape -- List with 1 or 3 elements describing the tile shape that will be used 

200 to save the columns to disk. (list) 

201 default: [0] 

202 options: [0] or [1] or [int,int,int]. When list has only one element, it should 

203 be either 0 or 1. When the list has three elements, they should be the 

204 number of correlations, channels, rows. 

205  

206  

207 --- Data selection parameters --- 

208 field -- Select field using field id(s) or field name(s). 

209 [run listobs to obtain the list iof d's or names] 

210 default: ''=all fields If field string is a non-negative 

211 integer, it is assumed to be a field index 

212 otherwise, it is assumed to be a field name 

213 field='0~2'; field ids 0,1,2 

214 field='0,4,5~7'; field ids 0,4,5,6,7 

215 field='3C286,3C295'; fields named 3C286 and 3C295 

216 field = '3,4C*'; field id 3, all names starting with 4C 

217  

218 spw -- Select spectral window/channels 

219 default: ''=all spectral windows and channels 

220 spw='0~2,4'; spectral windows 0,1,2,4 (all channels) 

221 spw='<2'; spectral windows less than 2 (i.e. 0,1) 

222 spw='0:5~61'; spw 0, channels 5 to 61 

223 spw='0,10,3:3~45'; spw 0,10 all channels, spw 3 - chans 3 to 45. 

224 spw='0~2:2~6'; spw 0,1,2 with channels 2 through 6 in each. 

225 spw = '*:3~64' channels 3 through 64 for all sp id's 

226 spw = ' :3~64' will NOT work. 

227  

228 NOTE: mstransform does not support multiple channel ranges per 

229 spectral window (';'). 

230  

231 scan -- Scan number range 

232 default: ''=all 

233  

234 antenna -- Select data based on antenna/baseline 

235 default: '' (all) 

236 Non-negative integers are assumed to be antenna indices, and 

237 anything else is taken as an antenna name. 

238  

239 examples: 

240 antenna='5&6': baseline between antenna index 5 and index 6. 

241 antenna='VA05&VA06': baseline between VLA antenna 5 and 6. 

242 antenna='5&6;7&8': baselines 5-6 and 7-8 

243 antenna='5': all baselines with antenna 5 

244 antenna='5,6,10': all baselines including antennas 5, 6, or 10 

245 antenna='5,6,10&': all baselines with *only* antennas 5, 6, or 

246 10. (cross-correlations only. Use && 

247 to include autocorrelations, and &&& 

248 to get only autocorrelations.) 

249 antenna='!ea03,ea12,ea17': all baselines except those that 

250 include EVLA antennas ea03, ea12, or 

251 ea17. 

252  

253 correlation -- Correlation types or expression. 

254 default: '' (all correlations) 

255 example: correlation='XX,YY' 

256  

257 timerange -- Select data based on time range: 

258 default: '' (all); examples, 

259 timerange = 'YYYY/MM/DD/hh:mm:ss~YYYY/MM/DD/hh:mm:ss' 

260 Note: if YYYY/MM/DD is missing date, timerange defaults to the 

261 first day in the dataset 

262 timerange='09:14:0~09:54:0' picks 40 min on first day 

263 timerange='25:00:00~27:30:00' picks 1 hr to 3 hr 30min 

264 on next day 

265 timerange='09:44:00' data within one integration of time 

266 timerange='>10:24:00' data after this time 

267  

268 array -- (Sub)array number range 

269 default: ''=all 

270  

271 uvrange -- Select data within uvrange (default units meters) 

272 default: ''=all; example: 

273 uvrange='0~1000klambda'; uvrange from 0-1000 kilo-lambda 

274 uvrange='>4klambda';uvranges greater than 4 kilo-lambda 

275 uvrange='0~1000km'; uvrange in kilometers 

276  

277 observation -- Select by observation ID(s) 

278 default: ''=all 

279  

280 feed -- Selection based on the feed - NOT IMPLEMENTED YET 

281 default: ''=all 

282  

283  

284 datacolumn -- Which data column to use for processing (case-insensitive). 

285 default: 'corrected'; example: datacolumn='data' 

286 options: 'data', 'model', 'corrected', 'all','float_data', 'lag_data', 

287 'float_data,data', 'lag_data,data'. 

288  

289 NOTE: 'all' = whichever of the above that are present. If the requested 

290 column does not exist, the task will exit with an error. 

291  

292 When datacolumn is set to either one of the values 'model','all', 

293 'data,model,corrected', a sub-parameter realmodelcol will be enabled. 

294 See description below. 

295  

296 realmodelcol -- Make real a virtual MODEL column. If set to True, a real MODEL_DATA 

297 column will be added to the output MS based on the existing SOURCE_MODEL 

298 column. 

299 default: False 

300  

301  

302 keepflags -- Keep completely flagged rows in the output or drop them. This has no 

303 effect on partially flagged rows. All of the channels and correlations 

304 of a row must be flagged for it to be droppable, and a row must be 

305 well defined to be keepable. 

306  

307 IMPORTANT: Regardless of this parameter, flagged data is never included in 

308 channel averaging. On the other hand, partially flagged rows will 

309 always be included in time averaging. The average value of the 

310 flagged data for averages containing ONLY flagged data in the relevant 

311 output channel will be written to the output with the corresponding 

312 flag set to True, while only unflagged data is used on averages where 

313 there is some unflagged data with the flag set to False. 

314  

315 default: True (keep completely flagged rows in the output) 

316  

317  

318 usewtspectrum -- Force creation of the columns WEIGHT_SPECTRUM and SIGMA_SPECTRUM in the 

319 output MS. This flag can be used to force the creation of 

320 WEIGHT_SPECTRUM and SIGMA_SPECTRUM when they are not present in the 

321 input MS. When set to true, the output WEIGHT_SPECTRUM column will be 

322 populated using the input WEIGHT column, such that each channel in the 

323 WEIGHT_SPECTRUM column will get WEIGHT/nChannels. Note that if the 

324 WEIGHT_SPECTRUM column is present in the input MS the columns will 

325 always be created in the output MS, regardless of the value of this 

326 parameter. 

327  

328 default: False 

329  

330  

331 --- SPW combination parameters --- 

332 combinespws -- Combine the input spws into a new output spw. 

333 default: False 

334  

335 NOTE: This option is only supported when the number of channels is the same for 

336 all the spws. Using this option with different numbers of channels for 

337 different spws will result in an error. 

338  

339 NOTE: If the SPWs have different polarization settings only the common 

340 polarizations will be output. For instance, if SPW 1 has XX XY YX YY and 

341 SPW 2 has XX YY, the output MS will have a single SPW with XX YY polarizations 

342  

343 NOTE: Whenever the data to be combined has different EXPOSURE values 

344 in the spectral windows, mstransform will use the WEIGHT_SPECTRUM 

345 for the combination. If WEIGHT_SPECTRUM is not available, it will 

346 use the values from the WEIGHT column. Each output channel is calculated 

347 using the following equation: 

348  

349 outputChannel_j = SUM(inputChannel_i*contributionFraction_i*inputWeightSpectrum_i) 

350 ------------------------------------------------------------------ 

351 SUM(contributionFraction_i*inputWeightSpectrum_i) 

352  

353  

354 --- Channel averaging parameters --- 

355 chanaverage -- Average data across channels. Partially flagged data is not be included in the average 

356 unless all data contributing to a given output channel is flagged. In this case, 

357 mstransform calculates the average of all flagged data, and writes it to the output MS 

358 with the corresponding flag set to true. If present, WEIGHT_SPECTRUM/SIGMA_SPECTRUM 

359 are used together with the channelized flags (FLAG), to compute a weighted average 

360 (using WEIGHT_SPECTRUM for CORRECTED_DATA and SIGMA_SPECTRUM for DATA). 

361 default: False 

362  

363 chanbin -- Bin width for channel average in number of input channels. 

364 If a list is given, each bin applies to one of the selected SPWs. 

365  

366 default: 1 => no channel averaging. 

367 options: (int) or [int] 

368 example: chanbin=[2,3] => average 2 channels of 1st selected 

369 spectral window and 3 in the second one. 

370  

371 NOTE: WEIGHT_SPECTRUM/SIGMA_SPECTRUM will be used (if present) in 

372 addition to the flags to compute a weighted average. The calculations 

373 is done as follows: 

374  

375 1) When WEIGHT_SPECTRUM/SIGMA_SPECTRUM are not present: 

376 Avg = SUM(Chan_i*Flag_i)/SUM(Flag_i) 

377  

378 2) When WEIGHT_SPECTRUM/SIGMA_SPECTRUM are present: 

379 Avg = SUM(Chan_i*Flag_i*WeightSpectrum_i)/SUM(Flag_i*WeightSpectrum_i) 

380  

381 If combinespws=True, then chanbin can only be a (int). This is because the 

382 channel average then refers to the already combined MS after spw combination, 

383 which has a single spw. 

384  

385  

386 --- Hanning smoothing parameters --- 

387 hanning -- Hanning smooth frequency channel data to remove Gibbs ringing. 

388 default: False 

389  

390 --- Regrid parameters --- 

391 regridms -- Transform channel labels and visibilities to a different spectral reference frame. 

392 Notice that u,v,w data is not transformed. 

393 default: False 

394  

395 mode -- Regridding mode. 

396 default: 'channel'; produces equidistant grid based on first selected channel. 

397 options: 'velocity', 'frequency', 'channel_b'. 

398  

399 When set to velocity or frequency, it means that the channels must be specified 

400 in the respective units. When set to channel_b it means an alternative 'channel' 

401 mode that does not force an equidistant grid. It is faster. 

402  

403 nchan -- Number of channels in the output spw (int). 

404 default: -1 

405  

406 start -- First channel to use in the output spw (depends on the mode) 

407 default: 0 --> when mode='channel' 

408  

409 When mode='channel', 'start' means the first channel in the input spw 

410 to use when creating the output spw. When mode='frequency' or mode='velocity', 

411 'start' means the frequency or velocity, respectively, of the first output 

412 channel. If this information is not available, leave it blank and mstransform 

413 will calculate it. 

414  

415 width -- Width of input channels that are used to create an output channel. 

416 default: 1 

417  

418 Note that mstransform will only shift spws with channel widths of the same 

419 sign in a single operation. If you are regridding spws with mixed positive 

420 and negative channel widths, you should run this task separated for each 

421 group of spws. You can verify the channel widths for your MS using 

422 listobs for example, and looking at the SPW table, column ChanWid. 

423  

424 nspw -- Number of output spws to create in the output MS/MMS (int). 

425 default: 1 --> it means, do not separate the spws. 

426  

427 One can regrid the MS or not and further separate the 

428 output into a given number of spws. Internally, the framework 

429 will combine the selected spws before separating them so that 

430 channel gaps and overlaps are taken into account. This parameter 

431 will create a regular grid of spws in the output MS. If nchan 

432 is set, it will refer to the number of output channels in each 

433 of the separated spws. 

434  

435 interpolation -- Spectral interpolation method. 

436 default: 'linear' 

437 options: 'nearest', 'cubic', 'spline', 'fftshift' 

438  

439 phasecenter -- Direction measure or fieldid. To be used in mosaics to indicate 

440 the center direction to be used in the spectral coordinate transformation. 

441 default: '' (first selected field) 

442 options: FIELD_ID (int) or center coordinate measure (str). 

443 example: phasecenter=6 or phasecenter='J2000 19h30m00 -40d00m00' 

444  

445 restfreq -- Specify rest frequency to use for output. 

446 default: ''; occasionally it is necessary to set this. 

447 example1 for some VLA spectral line data. 

448 example2 for NH_3 (1,1) put restfreq='23.694496GHz'. 

449  

450 outframe -- Output reference frame (case-insensitive). 

451 default: ''; it will keep the input reference frame. 

452 options: 'LSRK', 'LSRD', 'BARY', 'GALACTO', 'LGROUP', 'CMB', 'GEO', 'TOPO'. 

453  

454 veltype -- Definition of velocity (as used in mode). 

455 default: 'radio' 

456  

457 preaverage -- Pre-average channels before regridding. This is done when the ratio 

458 between the output and and input widths is greater than 2. This has 

459 been disabled since CASA 5.0 

460 default: False 

461  

462  

463 --- Time averaging parameters --- 

464 timeaverage -- Average data across time. Partially flagged data is not be included in the average 

465 unless all data contributing to a given output channel is flagged. In this case, 

466 mstransform calculates the average of all flagged data, and writes it to the output MS 

467 with the corresponding flag set to true. If keepflags=False, the fully flagged data 

468 is not be written to the output MS. If present, WEIGHT_SPECTRUM/SIGMA_SPECTRUM 

469 are used together with the channelized flags (FLAG), to compute a weighted average 

470 (using WEIGHT_SPECTRUM for CORRECTED_DATA and SIGMA_SPECTRUM for DATA). Otherwise 

471 WEIGHT/SIGMA are used instead to average together data from different integrations. 

472  

473 default: False 

474  

475 timebin -- Bin width for time average in seconds. 

476 default: '0s' 

477  

478 timespan -- Let the timebin span across scan, state or both. 

479 State is equivalent to sub-scans. One scan may have several 

480 state ids. For ALMA MSs, the sub-scans are limited to about 

481 30s duration each. In these cases, the task will automatically 

482 add state to the timespan parameter. To see the number of states 

483 in an MS, use the msmd tool. See help msmd. 

484  

485 default: '' (separate time bins by both of the above) 

486 options: 'scan', 'state', 'state,scan' 

487  

488 examples: 

489 timespan = 'scan'; can be useful when the scan number 

490 goes up with each integration as in many WSRT MSs. 

491 timespan = ['scan', 'state']: disregard scan and state 

492 numbers when time averaging. 

493 timespan = 'state,scan'; same as above. 

494  

495 maxuvwdistance -- Provide a maximum separation of start-to-end baselines 

496 that can be included in an average. (int) 

497 default: 0.0 (given in meters) 

498  

499 --- On-the-fly calibration parameters --- 

500 docallib -- Enable on-the-fly (OTF) calibration as in task applycal 

501 default: False 

502  

503 callib -- Path to calibration library file, which is a ascii file containing 

504 the parameters to correct the data as task in task applycal, namely 

505 gaintable/gainfield/interp/spwmap/calwt. In a Cal Library file, each 

506 row expresses the calibration apply instructions for a particular 

507 caltable and (optionally) a specific selection of data in the MS to 

508 which it is to be applied. 

509  

510 default: '' (there is no default callib file) 

511  

512 examples: 

513  

514 caltable='cal.G' tinterp='linear' calwt=True 

515  

516 -> Arrange a caltable called cal.G to be applied (with no detailed selection) 

517 to all MS data with linear interpolation in time, and with the weights also 

518 calibrated. 

519  

520 caltable='cal.G' tinterp='linear' fldmap='nearest' spwmap=[0,1,1,3] calwt=True 

521 caltable='cal.B' finterp='linear' fldmap='3' spwmap=[0,0,0,0] calwt=False 

522  

523 -> In this case, solutions from cal.G will be selected based on directional 

524 proximity ('nearest') for each MS field via the fldmap parameter, and spw 2 

525 will be calibrated by spw 1 solutions. For cal.B, solutions from field id 3 

526 will be used exclusively, with spw 0 calibrating all MS spws. 

527  

528  

529  

530  

531 ------ Multi-MS Processing and Heuristics --------- 

532  

533 ** Input Multi-MS (MMS) ** 

534  

535 Task mstransform will process an input MMS in parallel whenever possible. Each sub-MS of 

536 the MMS will be processed in a separate engine and the results will be post-processed at the 

537 end to create an output MMS. The output MMS will have the same separationaxis of the input 

538 MMS, which will be written to the table.info file inside the MMS directory. 

539  

540 Naturally, some transformations available in mstransform require more care when the user 

541 first partition the MS. If one wants to do a combination of spws by setting the parameter 

542 combinespws = True in mstransform, the input MMS needs to contain all the 

543 selected spws in each of the sub-MSs or the processing will fail. For this, one may set the initial 

544 separationaxis to scan or use the default auto with a proper numsubms set so that each sub- MS in 

545 the MMS is self-contained with all the necessary spws for the combination. 

546  

547 The task will check if the sub-MSs contain all the selected spws when combinespws=True 

548 and if not, it will issue a warning and process the input MMS as a monolithic MS. In this 

549 case, the separation axis of the output MMS will be set to scan, regardless of what the input 

550 axis was. 

551  

552 A similar case happens when the separation axis of the input MMS is per scan and the user 

553 asks to do time averaging with time spanning across scans. If the individual sub-MSs are not 

554 self-contained of the necessary scans and the duration of the scans is shorter than the given 

555 timebin, the spanning will not be possible. In this case, the task will process the input MMS as 

556 a monolithic MS and will set the axis of the output MMS to spw. 

557  

558 It is important that the user sets the separation axis correctly when first partitioning the MS. 

559 See the table below for when it is possible to process the input MMS in parallel or not, using 

560 mstransform. 

561  

562 input MMS axis combinespws=True nspw > 1 timeaverage=True, timespan='scan' 

563 ------------------------------------------------------------------------------- 

564 scan YES YES NO 

565 spw NO NO YES 

566 auto MAYBE MAYBE MAYBE 

567  

568  

569 ------ EXAMPLES ------ 

570  

571 More documentation on mstransform can be found here: 

572 http://www.eso.org/~scastro/ALMA/casa/MST/MSTransformDocs/MSTransformDocs.html 

573  

574 1) Split out a single channel. 

575 mstransform(vis='ctb80-vsm.ms', outputvis='mychn.ms', datacolumn='data', spw='0:25') 

576  

577 2) Only combine the selected spws into a single output spw. 

578 mstransform(vis='Four_ants.ms', outputvis='myspw.ms', combinespws=True, spw='0~3') 

579  

580 3) Combine two spws and regrid one field, using two input channels to make one output. 

581 mstransform(vis='jupiter6cm.demo.ms',outputvis='test1.ms',datacolumn='DATA',field='11', 

582 spw='0,1', combinespws=True, regridms=True, nchan=1, width=2) 

583  

584 4) Combine 24 spws and regrid in frequency mode to create 21 output channels. Change the 

585 phase center. 

586 mstransform(vis='g19_d2usb_targets_line.ms', outputvis='test2.ms', datacolumn='DATA', 

587 combinespws=True, regridms=True, mode='frequency', nchan=21, start='229587.0MHz', 

588 width='1600kHz', phasecenter="J2000 18h25m56.09 -12d04m28.20") 

589  

590 5) Only apply Hanning smoothing to MS. 

591 mstransform(vis='g19_d2usb_targets_line.ms', outputvis='test3.ms', datacolumn='DATA', 

592 hanning=True) 

593  

594 6) Change the reference frame and apply Hanning smoothing after combining all spws. 

595 mstransform(vis='g19_d2usb_targets_line.ms', outputvis='test4.ms', datacolumn='DATA', 

596 combinespws=True, regridms=True, mode="channel", outframe="BARY", 

597 phasecenter="J2000 18h25m56.09 -12d04m28.20", hanning = True) 

598  

599 7) Apply time averaging using a bin of 30 seconds on the default CORRECTED column. 

600 mstransform(vis='g19_d2usb_targets_line.ms', outputvis='test5.ms', timeaverage=True, 

601 timebin='30s') 

602  

603 8) Apply OTF calibration to ng5921 using a calibration library 

604 mstransform(vis='ngc5921.ms', outputvis='ngc5921_calibrated.ms',docallib=True, 

605 callib='unittest/mstransform/ngc5921_regression/ngc5921_callib.txt') 

606 

607 

608 """ 

609 

610 _info_group_ = """manipulation""" 

611 _info_desc_ = """Split the MS, combine/separate/regrid spws and do channel and time averaging""" 

612 

613 def __call__( self, vis='', outputvis='', createmms=False, separationaxis='auto', numsubms='auto', tileshape=[ int(0) ], field='', spw='', scan='', antenna='', correlation='', timerange='', intent='', array='', uvrange='', observation='', feed='', datacolumn='corrected', realmodelcol=False, keepflags=True, usewtspectrum=False, combinespws=False, chanaverage=False, chanbin=int(1), hanning=False, regridms=False, mode='channel', nchan=int(-1), start=int(0), width=int(1), nspw=int(1), interpolation='linear', phasecenter='', restfreq='', outframe='', veltype='radio', preaverage=False, timeaverage=False, timebin='0s', timespan='', maxuvwdistance=float(0.0), docallib=False, callib='', douvcontsub=False, fitspw='', fitorder=int(0), want_cont=False, denoising_lib=True, nthreads=int(1), niter=int(1), disableparallel=False, ddistart=int(-1), taql='', monolithic_processing=False, reindex=True ): 

614 schema = {'vis': {'type': 'cReqPath', 'coerce': _coerce.expand_path}, 'outputvis': {'type': 'cStr', 'coerce': _coerce.to_str}, 'createmms': {'type': 'cBool'}, 'separationaxis': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'baseline', 'AUTO', 'SPW', 'SCAN', 'auto', 'spw', 'BASELINE', 'scan' ]}, 'numsubms': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cInt'}]}, 'tileshape': {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}, 'field': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'spw': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'scan': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'antenna': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'correlation': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}]}, 'timerange': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'intent': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'array': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'uvrange': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'observation': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'feed': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}, {'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'datacolumn': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'DATA', 'model', 'corrected', 'LAG_DATA', 'lag_data', 'FLOAT_DATA,DATA', 'FLOAT_DATA', 'CORRECTED', 'lag_data,data', 'float_data', 'float_data,data', 'DATA,MODEL,CORRECTED', 'ALL', 'MODEL', 'all', 'data,model,corrected', 'LAG_DATA,DATA', 'data' ]}, 'realmodelcol': {'type': 'cBool'}, 'keepflags': {'type': 'cBool'}, 'usewtspectrum': {'type': 'cBool'}, 'combinespws': {'type': 'cBool'}, 'chanaverage': {'type': 'cBool'}, 'chanbin': {'anyof': [{'type': 'cInt'}, {'type': 'cIntVec', 'coerce': [_coerce.to_list,_coerce.to_intvec]}]}, 'hanning': {'type': 'cBool'}, 'regridms': {'type': 'cBool'}, 'mode': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'channel', 'velocity', 'frequency', 'channel_b' ]}, 'nchan': {'type': 'cInt'}, 'start': {'type': 'cVariant', 'coerce': [_coerce.to_variant]}, 'width': {'type': 'cVariant', 'coerce': [_coerce.to_variant]}, 'nspw': {'type': 'cInt'}, 'interpolation': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'nearest', 'linear', 'spline', 'fftshift', 'cubic' ]}, 'phasecenter': {'type': 'cVariant', 'coerce': [_coerce.to_variant]}, 'restfreq': {'type': 'cStr', 'coerce': _coerce.to_str}, 'outframe': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'SOURCE', 'LSRK', 'CMB', 'geo', 'source', 'bary', 'GALACTO', 'GEO', 'TOPO', 'LSRD', 'lsrk', 'topo', 'LGROUP', 'BARY', 'lsrd', 'cmb', 'galacto', 'lgroup', '' ]}, 'veltype': {'type': 'cStr', 'coerce': _coerce.to_str, 'allowed': [ 'optical', 'radio', 'OPTICAL', 'RADIO' ]}, 'preaverage': {'type': 'cBool'}, 'timeaverage': {'type': 'cBool'}, 'timebin': {'type': 'cStr', 'coerce': _coerce.to_str}, 'timespan': {'anyof': [{'type': 'cStr', 'coerce': _coerce.to_str}, {'type': 'cStrVec', 'coerce': [_coerce.to_list,_coerce.to_strvec]}]}, 'maxuvwdistance': {'type': 'cFloat', 'coerce': _coerce.to_float}, 'docallib': {'type': 'cBool'}, 'callib': {'type': 'cStr', 'coerce': _coerce.to_str}, 'douvcontsub': {'type': 'cBool'}, 'fitspw': {'type': 'cStr', 'coerce': _coerce.to_str}, 'fitorder': {'type': 'cInt'}, 'want_cont': {'type': 'cBool'}, 'denoising_lib': {'type': 'cBool'}, 'nthreads': {'type': 'cInt'}, 'niter': {'type': 'cInt'}, 'disableparallel': {'type': 'cBool'}, 'ddistart': {'type': 'cInt'}, 'taql': {'type': 'cStr', 'coerce': _coerce.to_str}, 'monolithic_processing': {'type': 'cBool'}, 'reindex': {'type': 'cBool'}} 

615 doc = {'vis': vis, 'outputvis': outputvis, 'createmms': createmms, 'separationaxis': separationaxis, 'numsubms': numsubms, 'tileshape': tileshape, 'field': field, 'spw': spw, 'scan': scan, 'antenna': antenna, 'correlation': correlation, 'timerange': timerange, 'intent': intent, 'array': array, 'uvrange': uvrange, 'observation': observation, 'feed': feed, 'datacolumn': datacolumn, 'realmodelcol': realmodelcol, 'keepflags': keepflags, 'usewtspectrum': usewtspectrum, 'combinespws': combinespws, 'chanaverage': chanaverage, 'chanbin': chanbin, 'hanning': hanning, 'regridms': regridms, 'mode': mode, 'nchan': nchan, 'start': start, 'width': width, 'nspw': nspw, 'interpolation': interpolation, 'phasecenter': phasecenter, 'restfreq': restfreq, 'outframe': outframe, 'veltype': veltype, 'preaverage': preaverage, 'timeaverage': timeaverage, 'timebin': timebin, 'timespan': timespan, 'maxuvwdistance': maxuvwdistance, 'docallib': docallib, 'callib': callib, 'douvcontsub': douvcontsub, 'fitspw': fitspw, 'fitorder': fitorder, 'want_cont': want_cont, 'denoising_lib': denoising_lib, 'nthreads': nthreads, 'niter': niter, 'disableparallel': disableparallel, 'ddistart': ddistart, 'taql': taql, 'monolithic_processing': monolithic_processing, 'reindex': reindex} 

616 assert _pc.validate(doc,schema), create_error_string(_pc.errors) 

617 _logging_state_ = _start_log( 'mstransform', [ 'vis=' + repr(_pc.document['vis']), 'outputvis=' + repr(_pc.document['outputvis']), 'createmms=' + repr(_pc.document['createmms']), 'separationaxis=' + repr(_pc.document['separationaxis']), 'numsubms=' + repr(_pc.document['numsubms']), 'tileshape=' + repr(_pc.document['tileshape']), 'field=' + repr(_pc.document['field']), 'spw=' + repr(_pc.document['spw']), 'scan=' + repr(_pc.document['scan']), 'antenna=' + repr(_pc.document['antenna']), 'correlation=' + repr(_pc.document['correlation']), 'timerange=' + repr(_pc.document['timerange']), 'intent=' + repr(_pc.document['intent']), 'array=' + repr(_pc.document['array']), 'uvrange=' + repr(_pc.document['uvrange']), 'observation=' + repr(_pc.document['observation']), 'feed=' + repr(_pc.document['feed']), 'datacolumn=' + repr(_pc.document['datacolumn']), 'realmodelcol=' + repr(_pc.document['realmodelcol']), 'keepflags=' + repr(_pc.document['keepflags']), 'usewtspectrum=' + repr(_pc.document['usewtspectrum']), 'combinespws=' + repr(_pc.document['combinespws']), 'chanaverage=' + repr(_pc.document['chanaverage']), 'chanbin=' + repr(_pc.document['chanbin']), 'hanning=' + repr(_pc.document['hanning']), 'regridms=' + repr(_pc.document['regridms']), 'mode=' + repr(_pc.document['mode']), 'nchan=' + repr(_pc.document['nchan']), 'start=' + repr(_pc.document['start']), 'width=' + repr(_pc.document['width']), 'nspw=' + repr(_pc.document['nspw']), 'interpolation=' + repr(_pc.document['interpolation']), 'phasecenter=' + repr(_pc.document['phasecenter']), 'restfreq=' + repr(_pc.document['restfreq']), 'outframe=' + repr(_pc.document['outframe']), 'veltype=' + repr(_pc.document['veltype']), 'preaverage=' + repr(_pc.document['preaverage']), 'timeaverage=' + repr(_pc.document['timeaverage']), 'timebin=' + repr(_pc.document['timebin']), 'timespan=' + repr(_pc.document['timespan']), 'maxuvwdistance=' + repr(_pc.document['maxuvwdistance']), 'docallib=' + repr(_pc.document['docallib']), 'callib=' + repr(_pc.document['callib']), 'douvcontsub=' + repr(_pc.document['douvcontsub']), 'fitspw=' + repr(_pc.document['fitspw']), 'fitorder=' + repr(_pc.document['fitorder']), 'want_cont=' + repr(_pc.document['want_cont']), 'denoising_lib=' + repr(_pc.document['denoising_lib']), 'nthreads=' + repr(_pc.document['nthreads']), 'niter=' + repr(_pc.document['niter']), 'disableparallel=' + repr(_pc.document['disableparallel']), 'ddistart=' + repr(_pc.document['ddistart']), 'taql=' + repr(_pc.document['taql']), 'monolithic_processing=' + repr(_pc.document['monolithic_processing']), 'reindex=' + repr(_pc.document['reindex']) ] ) 

618 task_result = None 

619 try: 

620 task_result = _mstransform_t( _pc.document['vis'], _pc.document['outputvis'], _pc.document['createmms'], _pc.document['separationaxis'], _pc.document['numsubms'], _pc.document['tileshape'], _pc.document['field'], _pc.document['spw'], _pc.document['scan'], _pc.document['antenna'], _pc.document['correlation'], _pc.document['timerange'], _pc.document['intent'], _pc.document['array'], _pc.document['uvrange'], _pc.document['observation'], _pc.document['feed'], _pc.document['datacolumn'], _pc.document['realmodelcol'], _pc.document['keepflags'], _pc.document['usewtspectrum'], _pc.document['combinespws'], _pc.document['chanaverage'], _pc.document['chanbin'], _pc.document['hanning'], _pc.document['regridms'], _pc.document['mode'], _pc.document['nchan'], _pc.document['start'], _pc.document['width'], _pc.document['nspw'], _pc.document['interpolation'], _pc.document['phasecenter'], _pc.document['restfreq'], _pc.document['outframe'], _pc.document['veltype'], _pc.document['preaverage'], _pc.document['timeaverage'], _pc.document['timebin'], _pc.document['timespan'], _pc.document['maxuvwdistance'], _pc.document['docallib'], _pc.document['callib'], _pc.document['douvcontsub'], _pc.document['fitspw'], _pc.document['fitorder'], _pc.document['want_cont'], _pc.document['denoising_lib'], _pc.document['nthreads'], _pc.document['niter'], _pc.document['disableparallel'], _pc.document['ddistart'], _pc.document['taql'], _pc.document['monolithic_processing'], _pc.document['reindex'] ) 

621 except Exception as exc: 

622 _except_log('mstransform', exc) 

623 raise 

624 finally: 

625 task_result = _end_log( _logging_state_, 'mstransform', task_result ) 

626 return task_result 

627 

628mstransform = _mstransform( ) 

629