Difference: AmpelVariables (1 vs. 2)

AMPEL Variables

When you're designing filters for AMPEL, it's good to know what variables are available for use. In general, AMPEL can call all of the information distributed in the Avro alerts for a given transient, as well as fast calculations you can define that make use of these values. Examples of derived quantities could be the maximum observed brightness in g or r band, or the mean colour of the transient.

In addition, subject to some limitations, AMPEL can cross-match with pre-defined catalogues. In general, the limitation for us is computational resources for a filter. If you can effectively pre-filter transients, then you can mostly match without restriction to pre-defined catalogues. It is easiest to consider binary statements (does this alert match, within a radius of your choice, any entries in a catalogue). However, you can also make use of any information contained in a catalogue, such as source class or redshift.

We are developing a photo-z estimator, and a number of other template fitting packages. These variables will be available in AMPEL when they are completed, but we do not currently have a timescale for these. So, for now, do not include them when designing filters.

Avro Alert Field Names

['jd', 'fid', 'pid', 'diffmaglim', 'pdiffimfilename', 'programpi', 'programid', 'candid', 'isdiffpos', 'tblid', 'nid', 'rcid', 'field', 'xpos', 'ypos', 'ra', 'dec', 'magpsf', 'sigmapsf', 'chipsf', 'magap', 'sigmagap', 'distnr', 'magnr', 'sigmagnr', 'chinr', 'sharpnr', 'sky', 'magdiff', 'fwhm', 'classtar', 'mindtoedge', 'magfromlim', 'seeratio', 'aimage', 'bimage', 'aimagerat', 'bimagerat', 'elong', 'nneg', 'nbad', 'rb', 'ssdistnr', 'ssmagnr', 'ssnamenr', 'sumrat', 'magapbig', 'sigmagapbig', 'ranr', 'decnr', 'sgmag', 'srmag', 'simag', 'szmag', 'sgscore', 'distpsnr1', 'ndethist', 'ncovhist', 'jdstarthist', 'jdendhist']

Full Avro Schema Description

A full description of each field is given below:

{'doc': 'avro alert schema',
 'fields': [{'doc': 'add descriptions like this',
   'name': 'objectId',
   'type': 'string'},
  {'name': 'candid', 'type': 'long'},
  {'name': 'candidate',
   'type': {'doc': 'avro alert schema',
    'fields': [{'doc': 'Observation Julian date at start of exposure [days]',
      'name': 'jd',
      'type': 'double'},
     {'doc': 'Filter ID (1=g; 2=R; 3=i)', 'name': 'fid', 'type': 'int'},
     {'doc': 'Processing ID for image', 'name': 'pid', 'type': 'long'},
     {'default': None,
      'doc': '5-sigma mag limit in difference image based on PSF-fit photometry [mag]',
      'name': 'diffmaglim',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'filename of positive (sci minus ref) difference image',
      'name': 'pdiffimfilename',
      'type': ['string', 'null']},
     {'default': None,
      'doc': 'Principal investigator attached to program ID',
      'name': 'programpi',
      'type': ['string', 'null']},
     {'doc': 'Program ID: encodes either public, collab, or caltech mode',
      'name': 'programid',
      'type': 'int'},
     {'doc': 'Candidate ID from operations DB',
      'name': 'candid',
      'type': 'long'},
     {'doc': 't or 1 => candidate is from positive (sci minus ref) subtraction; f or 0 => candidate is from negative (ref minus sci) subtraction',
      'name': 'isdiffpos',
      'type': 'string'},
     {'default': None,
      'doc': 'Internal pipeline table extraction ID',
      'name': 'tblid',
      'type': ['long', 'null']},
     {'default': None,
      'doc': 'Night ID',
      'name': 'nid',
      'type': ['int', 'null']},
     {'default': None,
      'doc': 'Readout channel ID [00 .. 63]',
      'name': 'rcid',
      'type': ['int', 'null']},
     {'default': None,
      'doc': 'ZTF field ID',
      'name': 'field',
      'type': ['int', 'null']},
     {'default': None,
      'doc': 'x-image position of candidate [pixels]',
      'name': 'xpos',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'y-image position of candidate [pixels]',
      'name': 'ypos',
      'type': ['float', 'null']},
     {'doc': 'Right Ascension of candidate; J2000 [deg]',
      'name': 'ra',
      'type': 'double'},
     {'doc': 'Declination of candidate; J2000 [deg]',
      'name': 'dec',
      'type': 'double'},
     {'doc': 'magnitude from PSF-fit photometry [mag]',
      'name': 'magpsf',
      'type': 'float'},
     {'doc': '1-sigma uncertainty in magpsf [mag]',
      'name': 'sigmapsf',
      'type': 'float'},
     {'default': None,
      'doc': 'Reduced chi-square for PSF-fit',
      'name': 'chipsf',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Aperture mag using 8 pixel diameter aperture [mag]',
      'name': 'magap',
      'type': ['float', 'null']},
     {'default': None,
      'doc': '1-sigma uncertainty in magap [mag]',
      'name': 'sigmagap',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'distance to nearest source in reference image PSF-catalog [pixels]',
      'name': 'distnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'magnitude of nearest source in reference image PSF-catalog [mag]',
      'name': 'magnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': '1-sigma uncertainty in magnr [mag]',
      'name': 'sigmagnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'DAOPhot chi parameter of nearest source in reference image PSF-catalog',
      'name': 'chinr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'DAOPhot sharp parameter of nearest source in reference image PSF-catalog',
      'name': 'sharpnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Local sky background estimate [DN]',
      'name': 'sky',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Difference: magap - magpsf [mag]',
      'name': 'magdiff',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Full Width Half Max assuming a Gaussian core, from SExtractor [pixels]',
      'name': 'fwhm',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Star/Galaxy classification score from SExtractor',
      'name': 'classtar',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Distance to nearest edge in image [pixels]',
      'name': 'mindtoedge',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Difference: diffmaglim - magap [mag]',
      'name': 'magfromlim',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Ratio: difffwhm / fwhm',
      'name': 'seeratio',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Windowed profile RMS afloat major axis from SExtractor [pixels]',
      'name': 'aimage',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Windowed profile RMS afloat minor axis from SExtractor [pixels]',
      'name': 'bimage',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Ratio: aimage / fwhm',
      'name': 'aimagerat',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Ratio: bimage / fwhm',
      'name': 'bimagerat',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Ratio: aimage / bimage',
      'name': 'elong',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'number of negative pixels in a 5 x 5 pixel stamp',
      'name': 'nneg',
      'type': ['int', 'null']},
     {'default': None,
      'doc': 'number of prior-tagged bad pixels in a 5 x 5 pixel stamp',
      'name': 'nbad',
      'type': ['int', 'null']},
     {'default': None,
      'doc': 'RealBogus quality score; range is 0 to 1 where closer to 1 is more reliable',
      'name': 'rb',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'distance to nearest known solar system object if exists within 30 arcsec [arcsec]',
      'name': 'ssdistnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'magnitude of nearest known solar system object if exists within 30 arcsec (usually V-band from MPC archive) [mag]',
      'name': 'ssmagnr',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'name of nearest known solar system object if exists within 30 arcsec (from MPC archive)',
      'name': 'ssnamenr',
      'type': ['string', 'null']},
     {'default': None,
      'doc': 'Ratio: sum(pixels) / sum(|pixels|) in a 5 x 5 pixel stamp where stamp is first median-filtered to mitigate outliers',
      'name': 'sumrat',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Aperture mag using 18 pixel diameter aperture [mag]',
      'name': 'magapbig',
      'type': ['float', 'null']},
     {'default': None,
      'doc': '1-sigma uncertainty in magapbig [mag]',
      'name': 'sigmagapbig',
      'type': ['float', 'null']},
     {'doc': 'Right Ascension of nearest source in reference image PSF-catalog; J2000 [deg]',
      'name': 'ranr',
      'type': 'double'},
     {'doc': 'Declination of nearest source in reference image PSF-catalog; J2000 [deg]',
      'name': 'decnr',
      'type': 'double'},
     {'default': None,
      'doc': 'g-band magnitude of closest source from PS1 catalog; if exists within 30 arcsec [mag]',
      'name': 'sgmag',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'r-band magnitude of closest source from PS1 catalog; if exists within 30 arcsec [mag]',
      'name': 'srmag',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'i-band magnitude of closest source from PS1 catalog; if exists within 30 arcsec [mag]',
      'name': 'simag',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'z-band magnitude of closest source from PS1 catalog; if exists within 30 arcsec [mag]',
      'name': 'szmag',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Star/Galaxy score of closest source from PS1 catalog; if exists within 30 arcsec: 0 <= sgscore <= 1 where closer to 1 implies higher likelihood of being a star',
      'name': 'sgscore',
      'type': ['float', 'null']},
     {'default': None,
      'doc': 'Distance of closest source from PS1 catalog; if exists within 30 arcsec [arcsec]',
      'name': 'distpsnr1',
      'type': ['float', 'null']},
     {'doc': 'Number of spatially-coincident detections falling within 1.5 arcsec going back to beginning of survey; only detections that fell on the same field and readout-channel ID where the input candidate was observed are counted',
      'name': 'ndethist',
      'type': 'int'},
     {'doc': 'Number of times input candidate position fell on any field and readout-channel going back to beginning of survey',
      'name': 'ncovhist',
      'type': 'int'},
     {'default': None,
      'doc': 'Earliest Julian date of epoch corresponding to ndethist [days]',
      'name': 'jdstarthist',
      'type': ['double', 'null']},
     {'default': None,
      'doc': 'Latest Julian date of epoch corresponding to ndethist [days]',
      'name': 'jdendhist',
      'type': ['double', 'null']}],
    'name': 'candidate',
    'namespace': 'ztf.alert',
    'type': 'record',
    'version': '1.5'}},
  {'default': None,
   'name': 'prv_candidates',
   'type': [{'items': {'doc': 'avro alert schema',
      'fields': [{'doc': 'Observation Julian date at start of exposure [days]',
        'name': 'jd',
        'type': 'double'},
       {'doc': 'Filter ID (1=g; 2=R; 3=i)', 'name': 'fid', 'type': 'int'},
       {'doc': 'Processing ID for image', 'name': 'pid', 'type': 'long'},
       {'default': None,
        'doc': '5-sigma mag limit in difference image based on PSF-fit photometry [mag]',
        'name': 'diffmaglim',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'filename of positive (sci minus ref) difference image',
        'name': 'pdiffimfilename',
        'type': ['string', 'null']},
       {'default': None,
        'doc': 'Principal investigator attached to program ID',
        'name': 'programpi',
        'type': ['string', 'null']},
       {'doc': 'Program ID: encodes either public, collab, or caltech mode',
        'name': 'programid',
        'type': 'int'},
       {'doc': 'Candidate ID from operations DB',
        'name': 'candid',
        'type': ['long', 'null']},
       {'doc': 't or 1 => candidate is from positive (sci minus ref) subtraction; f or 0 => candidate is from negative (ref minus sci) subtraction',
        'name': 'isdiffpos',
        'type': ['string', 'null']},
       {'default': None,
        'doc': 'Internal pipeline table extraction ID',
        'name': 'tblid',
        'type': ['long', 'null']},
       {'default': None,
        'doc': 'Night ID',
        'name': 'nid',
        'type': ['int', 'null']},
       {'default': None,
        'doc': 'Readout channel ID [00 .. 63]',
        'name': 'rcid',
        'type': ['int', 'null']},
       {'default': None,
        'doc': 'ZTF field ID',
        'name': 'field',
        'type': ['int', 'null']},
       {'default': None,
        'doc': 'x-image position of candidate [pixels]',
        'name': 'xpos',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'y-image position of candidate [pixels]',
        'name': 'ypos',
        'type': ['float', 'null']},
       {'doc': 'Right Ascension of candidate; J2000 [deg]',
        'name': 'ra',
        'type': ['double', 'null']},
       {'doc': 'Declination of candidate; J2000 [deg]',
        'name': 'dec',
        'type': ['double', 'null']},
       {'doc': 'magnitude from PSF-fit photometry [mag]',
        'name': 'magpsf',
        'type': ['float', 'null']},
       {'doc': '1-sigma uncertainty in magpsf [mag]',
        'name': 'sigmapsf',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Reduced chi-square for PSF-fit',
        'name': 'chipsf',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Aperture mag using 8 pixel diameter aperture [mag]',
        'name': 'magap',
        'type': ['float', 'null']},
       {'default': None,
        'doc': '1-sigma uncertainty in magap [mag]',
        'name': 'sigmagap',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'distance to nearest source in reference image PSF-catalog [pixels]',
        'name': 'distnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'magnitude of nearest source in reference image PSF-catalog [mag]',
        'name': 'magnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': '1-sigma uncertainty in magnr [mag]',
        'name': 'sigmagnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'DAOPhot chi parameter of nearest source in reference image PSF-catalog',
        'name': 'chinr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'DAOPhot sharp parameter of nearest source in reference image PSF-catalog',
        'name': 'sharpnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Local sky background estimate [DN]',
        'name': 'sky',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Difference: magap - magpsf [mag]',
        'name': 'magdiff',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Full Width Half Max assuming a Gaussian core, from SExtractor [pixels]',
        'name': 'fwhm',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Star/Galaxy classification score from SExtractor',
        'name': 'classtar',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Distance to nearest edge in image [pixels]',
        'name': 'mindtoedge',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Difference: diffmaglim - magap [mag]',
        'name': 'magfromlim',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Ratio: difffwhm / fwhm',
        'name': 'seeratio',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Windowed profile RMS afloat major axis from SExtractor [pixels]',
        'name': 'aimage',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Windowed profile RMS afloat minor axis from SExtractor [pixels]',
        'name': 'bimage',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Ratio: aimage / fwhm',
        'name': 'aimagerat',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Ratio: bimage / fwhm',
        'name': 'bimagerat',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Ratio: aimage / bimage',
        'name': 'elong',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'number of negative pixels in a 5 x 5 pixel stamp',
        'name': 'nneg',
        'type': ['int', 'null']},
       {'default': None,
        'doc': 'number of prior-tagged bad pixels in a 5 x 5 pixel stamp',
        'name': 'nbad',
        'type': ['int', 'null']},
       {'default': None,
        'doc': 'RealBogus quality score; range is 0 to 1 where closer to 1 is more reliable',
        'name': 'rb',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'distance to nearest known solar system object if exists within 30 arcsec [arcsec]',
        'name': 'ssdistnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'magnitude of nearest known solar system object if exists within 30 arcsec (usually V-band from MPC archive) [mag]',
        'name': 'ssmagnr',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'name of nearest known solar system object if exists within 30 arcsec (from MPC archive)',
        'name': 'ssnamenr',
        'type': ['string', 'null']},
       {'default': None,
        'doc': 'Ratio: sum(pixels) / sum(|pixels|) in a 5 x 5 pixel stamp where stamp is first median-filtered to mitigate outliers',
        'name': 'sumrat',
        'type': ['float', 'null']},
       {'default': None,
        'doc': 'Aperture mag using 18 pixel diameter aperture [mag]',
        'name': 'magapbig',
        'type': ['float', 'null']},
       {'default': None,
        'doc': '1-sigma uncertainty in magapbig [mag]',
        'name': 'sigmagapbig',
        'type': ['float', 'null']},
       {'doc': 'Right Ascension of nearest source in reference image PSF-catalog; J2000 [deg]',
        'name': 'ranr',
        'type': ['double', 'null']},
       {'doc': 'Declination of nearest source in reference image PSF-catalog; J2000 [deg]',
        'name': 'decnr',
        'type': ['double', 'null']}],
      'name': 'prv_candidate',
      'namespace': 'ztf.alert',
      'type': 'record',
      'version': '1.5'},
     'type': 'array'},
    'null']},
  {'default': None,
   'name': 'cutoutScience',
   'type': [{'doc': 'avro alert schema',
     'fields': [{'name': 'fileName', 'type': 'string'},
      {'doc': 'jpeg', 'name': 'stampData', 'type': 'bytes'}],
     'name': 'cutout',
     'namespace': 'ztf.alert',
     'type': 'record',
     'version': '1.5'},
    'null']},
  {'default': None,
   'name': 'cutoutTemplate',
   'type': ['ztf.alert.cutout', 'null']},
  {'default': None,
   'name': 'cutoutDifference',
   'type': ['ztf.alert.cutout', 'null']}],
 'name': 'alert',
 'namespace': 'ztf',
 'type': 'record',
 'version': '1.5'}

-- RobertStein - 23 Feb 2018