«Cambridge Astronomical Survey Unit Report and Future Programme (2008-2013) CONFIDENTIAL: JeS(07)xx i Cambridge Astronomical Survey Unit (CASU) Report ...»
Report and Future Programme
CONFIDENTIAL: JeS(07)xx i
Cambridge Astronomical Survey Unit (CASU)
Report and Future Program (2008–2013)
Observational astronomy in the UK is in a strong position with the advent of the pioneering near-infrared
(NIR) survey facilities of WFCAM and VISTA and the opportunities for detailed followup provided by membership of ESO. Signiﬁcant advances in astronomy have always relied heavily on surveys of the sky from radio, through optical to X-ray wavelengths and the new era of deep NIR surveys is the latest stage in this progression. It is also one where the UK has both a substantial lead and the requisite expertise in the Cambridge and Edinburgh survey units to exploit this advance in wide ﬁeld astronomy. Recognising the beneﬁts of this, ESO are now also heavily committed to public surveys through the VST and VISTA telescopes and to wider exploitation of expertly processed and archived science data products.
Members of the Cambridge Astronomy Survey Unit (CASU) have played a leading role in survey astronomy, not only by pioneering techniques to optimally extract knowledge from survey data, but also by taking a proactive role in exploiting this information to produce world-leading research. This synergy and feedback between data processing and exploitation is crucial. It has been a deliberate strategy that has provided the main strength and motivation of the group.
In the modern era wide-ﬁeld digital survey cameras produce enormous volumes of data that are way beyond the resource capacity and analysis skills of non-specialist astronomers. Systematic pipeline processing, calibration and legacy curation of observational data are a fundamental requirement of an end-to-end integrated observing strategy and a crucial component of a global Virtual Observatory. The CASU facility has been developed to allow an optimal ergonomic solution to this avalanche of data, through access to multiTB data storage systems and expert pipeline processing systems. Continuing development of the CASU processing and analysis pipelines will not only beneﬁt the UK astronomy community now, but will also be relevant in the era of Extremely Large Telescopes and the radio Square Kilometer Array, by developing the infrastructure to analyse the data from these facilities.
This rolling grant proposal builds on the tremendous advances already made and requests funding for CASU for the period 2008-2013 for the following activities: continued operation andmaintenance of the WFCAM data processing and calibration pipeline; advanced development, enhancement, and operation of the UK VISTA pipelines; maintenance and upgrades for the VDFS ESO pipeline deliverables; operational support and pipeline processing for the UK-led VST public surveys in the southern hemisphere and the equivalent precursor surveys in the northern hemisphere; support to provide a range of science data products and services (utilising the AstroGrid infrastructure) as a UK contribution to the Virtual Observatory initiative; continuation of support for maintaining and developing the UK ground-based telescope archives; and a modest research component for the research-active members of the group.
For completeness and context we also summarise the various CASU activities over the period of the current grant, which includes more than two years of operating a highly successful WFCAM pipeline. We also give a brief description of the progress made with the development of the VISTA data processing pipelines since these are also central to this grant application.
CONFIDENTIAL: JeS(07)xx ii Contents
1 Introduction The Cambridge Astronomical Survey Unit (CASU) is at the forefront of UK efforts to maximally exploit existing and upcoming survey-astronomy facilities such as WFCAM on UKIRT and the ESO VST and VISTA survey telescopes. These facilities dominate the landscape of ground-based survey work, offering almost limitless astronomical opportunities both in their own right and as essential components for the proper exploitation of the UK’s access to current 8–10m class facilities such as ESO, Gemini, Subaru and Keck, and to the future design of surveys to exploit the next generation ELT facilities.
The requirements to fully capitalise on WFCAM and VISTA, in particular, are extremely challenging due to the huge volume of data (≈100 TB/year) that must be ingested, processed, calibrated, curated and served automatically to the science community (see ﬁgure 1). However, the potential beneﬁt to UK astronomy is enormous.
Processing NIR survey data is technically much more demanding than the equivalent optical survey data. This stems from the fact that NIR detectors are inherently more unstable than their optical counterparts; the sky emission, roughly 100 times brighter than most objects of interest, varies in a complex spatial and temporal manner; and all of this is compounded by the large data volume that arises as a consequence of the bright background.
Over the past few years the survey aspects of frontier astronomy have changed dramatically and have become dominated by ground-based wide ﬁeld mosaic CCD and NIR imaging systems. In combination with systematic surveys in other wavebands ranging from the radio e.g. HIPASS to the x-ray e.g. XMM this has lead to a paradigm shift in the way survey astronomy is perceived.
Few users can handle either the data volumes and complexities of processing and calibrating optical and NIR mosaic camera data at their home institutions, or the related effort required to cross-federate data taken in different wavebands. With the realisation that specialist expertise focussed at a few institutions is an effective and powerful ergonomic solution to this problem, user expectations have now changed. The beneﬁts of end-to-end data ﬂow systems that cater for the vast majority of astronomical science requirements by providing expertly processed, calibrated and quality-controlled science products are obvious. Expectations change quickly and now focus on science exploitation, effectively taking the processing and calibration system for granted.
Related to this, is the shift toward queue-scheduled observing which is an effective way of designing and controlling the data taking process. This all helps to make semi-automatic end-to-end data processing architectures and data curation possible and helps ensure efﬁcient and optimal exploitation. Coupled with the crossdisciplinary nature of modern astronomy this has lead to ever increasing use of archival federated multiwaveband material, and the push, through GRID-enabled interlinked databases, toward a Global Virtual Observatory (VO).
In the VO context CASU works closely with the active AstroGrid VO group at the IoA, feeding requirements from the resource provider community to that project, and beneﬁting from early exposure to the AstroGrid VO infrastructure. Thus the Wide Field survey images at CASU were the ﬁrst signiﬁcant data collection to be made available through AstroGrid. CASU has also led the way in utilising AstroGrid acess technology to signiﬁcantly reduce the cost of publishing new data sets.
The Cambridge Astronomy Survey Unit has been at the forefront of processing and archiving digital wide-ﬁeld surveys for some time. We have used the experience gained from these earlier surveys (e.g. APM/SuperCOSMOS surveys, INT WFS and CIRSI) to design and build end-to-end processing systems for dealing with WFCAM and VISTA data as part of the VISTA data ﬂow system (VDFS) project. In parallel with this development, we have also been developing and running pipeline processing systems for several other existing wide ﬁeld mosaic cameras to enhance the pipeline components, to assess the problems of day-to-day running of operational pipelines and to explore the practicalities of controlling optional further processing stages driven from a survey quality control database.1 1 see, for example, the WFC processed data interface at http://apm2.ast.cam.ac.uk/cgi-bin/wfs/dqc.cgi and the WFCAM pipeline CONFIDENTIAL: JeS(07)xx 2 Figure 1: The huge growth in data volume from the WFCAM and VISTA surveys compared to current stateof-the-art surveys like 2MASS and SDSS.
2 Key Programme Aims CASU was formed in October 1998 following negotiations between the University of Cambridge and PPARC during the closure of the Royal Greenwich Observatory. The original core activities of CASU were funded from a rolling grant and were concerned with continuing to support and develop ground- and space-based UK astronomy wide-ﬁeld survey activities that had existed within the RGO. Over the intervening years various aspects of these core activities are now based on project-speciﬁc funding such as the development of the VISTA and GAIA data ﬂow systems (VDFS and GDFS). GAIA is outside the scope of this grant application 2 but the operational aspects of the VDFS are a central component.
The VDFS development grants (including a 6 month extension) cover the period to the end of March 2008 when it is anticipated that VISTA will begin on-sky survey science operations. The VDFS project includes various ESO deliverables in addition to prototyping the VISTA data ﬂow development using WFCAM data. However, it does not cover operational costs, the inevitable further development and maintenance that will be necessary when real on-sky data becomes available, or the necessary feedback and interaction with external users.
In this grant we are seeking funding for the period 2008–2013 for the following key activities:
• VISTA: this activity includes two major complementary strands:
– VISTA Science Products: advanced development, enhancement, maintenance and operation of the UK VISTA science processing pipelines for provision of calibrated science-ready data products to the UK community via the VISTA science archive including user liaison;
– VISTA Pipeline: effort for maintenance and upgrades to software and documentation for the VDFS ESO pipeline deliverables, this includes the VISTA calibration pipeline software modules to be run progress status and quality control web pages at http://casu.ast.cam.ac.uk/surveys-projects/wfcam/data-processing/.
2 We note that there are strong synergies at CASU between the GAIA activities and those presented here, with a number of staff
at Garching within the CPL environment and the Paranal summit pipeline for near-time quality control monitoring run within the ESO VLT operating environment;
• WFCAM: the current CASU rolling grant includes management, development, operation and maintenance of all WFCAM-related data processing. We are seeking continued funding at the same level to cover further operational development as needed, upgrades in response to changing demands, maintenance and enhancements to the data processing and calibration system and pipelines, and general user liaison;
• Science Service Centre: the provision of key science data products to the UK community, representing a signiﬁcant resource provider of content to the UK VO, operation and further development of pipeline processing and global calibration of data for science exploitation from optical wide ﬁeld survey imaging systems, including UK-led VST public surveys in the southern hemisphere and the corresponding optical surveys (e.g. IPHAS3 ) in the northern hemisphere;
• Data Archive Centre: management, operation, maintenance and further development of ING, UKIRT and AAT on-line archives, in particular developing more efﬁcient calibration schemes, progressing demanddriven on-the-ﬂy processing of data from these archives, and liaising closely with the AstroGrid project to ensure grid-enabled deliverables;
• Research allocation for research-active staff within the group to ensure rapid exploitation of survey products and to facilitate collaborative ventures with external groups;
Progress on each of the main activity areas, including future plans where appropriate, are detailed in the following sections. We note that, unlike the typical ’research only’ rolling grant programmes where PDRAs are allocated to single research themes, here the key CASU staff typically allocate their expertise across a number of the activity areas outlined here. The contribution of staff effort to each activity is noted in the relevant sections and summarised in the attached Appendices. Management of the programme is led by Irwin, assisted by Walton (science services and the Gaia/VO interfaces) and Bunclark (Vista operations).
3 Science Research: Highlights & Future Plans Several staff involved with CASU are active research workers. We therefore request an annual allocation of 1.5 FTE/yr of research effort to ensure the continuing synergy between active research workers and the collective national support facilities and survey projects we are operating. This effort will be split among ﬁve named individuals (Irwin, Gonzalez-Solares, Riello, Hodgkin & Greimel – 0.3 FTE/yr each) for science research capitalising on survey science in the areas presented in the following sections. 4
3.1 Near-ﬁeld Cosmology in the Local Group The discovery of the Sagittarius dwarf (Ibata, Gilmore, & Irwin, 1994, Nature, 370, 194), a satellite galaxy of the Milky Way caught in the act of tidal destruction, irrevocably changed our opinion of the local universe and brought modern cosmological predictions to the forefront of our Galactic backyard.
Since then Irwin has been involved in the discovery and analysis of numerous examples of new satellite galaxies and tidal streams and debris, in and around Local Group galaxies, including: great circles of carbon stars ripped from the Sagittarius Dwarf tracing out its previous orbits (e.g. Totten, Irwin & Whitelock 2000, MNRAS 314, 630); the discovery of another disrupting dwarf galaxy in Canis Major, this time embedded in the outer disk 3 IPHAS is the INT WFC Photometric Hα Survey of the Northern Galactic Plane, −5 o b +5o imaged in narrow-band Hα, and in Sloan r and i, at a spatial resolution of ∼1 arcsec to a 10σ magnitude limit r = 20 (Drew et al. 2005, MNRAS 362, 753). This survey is due for completion during the 2007/2008 observing season.
4 For this rolling grant submission we have been given permission to include a research component, as in previous CASU grants.