Next Article in Journal / Special Issue
Combining Faraday Tomography and Wavelet Analysis
Previous Article in Journal
Binary Neutron Star and Short Gamma-Ray Burst Simulations in Light of GW170817
Previous Article in Special Issue
Strategy to Explore Magnetized Cosmic Web with Forthcoming Large Surveys of Rotation Measure
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Science Pipelines for the Square Kilometre Array

Oxford e-Research Centre (OeRC), Department of Engineering Science, University of Oxford, Oxford OX1 3QG, UK
Author to whom correspondence should be addressed.
Galaxies 2018, 6(4), 120;
Received: 5 October 2018 / Revised: 14 November 2018 / Accepted: 15 November 2018 / Published: 20 November 2018
(This article belongs to the Special Issue The Power of Faraday Tomography)


The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of five zettabytes of data per year. A critical task for the SKA will be its ability to process data for science, which will need to be conducted by science pipelines. Together with polarization data from the LOFAR Multifrequency Snapshot Sky Survey (MSSS), we have been developing a realistic SKA-like science pipeline that can handle the large data volumes generated by LOFAR at 150 MHz. The pipeline uses task-based parallelism to image, detect sources and perform Faraday tomography across the entire LOFAR sky. The project thereby provides a unique opportunity to contribute to the technological development of the SKA telescope, while simultaneously enabling cutting-edge scientific results. In this paper, we provide an update on current efforts to develop a science pipeline that can enable tight constraints on the magnetised large-scale structure of the Universe.

1. Introduction

The Square Kilometre Array (SKA) will be the largest radio telescope ever constructed. The completed telescope will span two continents, with the project being jointly located in South Africa, where hundreds of dishes will be built, and in Australia, where 100,000 dipole antennas will be situated. Through the technique of interferometry, the signals from these antennas can be combined so that the array behaves like a single dish that is 150 km in diameter.
The SKA Organisation itself is located next to Jodrell Bank Observatory in Manchester, with the U.K. community being led by Manchester, Oxford and Cambridge, together with U.K. industrial partners. The U.K., via the Science and Technology Facilities Council (STFC), is contributing £100M towards construction of the first phase of the project. The key participating nations within the project include Australia, Canada, China, France, India, Italy, New Zealand, South Africa, Spain, Sweden, the Netherlands and the U.K. Further countries have expressed interest in joining the SKA Organisation, which will continue to expand over the coming years.
Both of the SKA telescope sites are ideal locations in which to situate a radio telescope; see Figure 1. The Australian site is located in the Western Australian outback, approximately 800 km from Perth, within a government-protected radio-quiet zone. This radio-quiet zone spans an area of ≈70,685 km 2 , which is approximately equivalent to twice the area of the island of Kyushu in Japan or about half the area of England in the U.K., but with a population of less than ∼100 people. This remote location is highly advantageous for radio astronomy, as humans are radio-loud with mobile phones, microwaves and various other emitters of radio frequency interference (RFI). All of these modern radio transmitters can dominate the tiny radio signals to which the telescope is sensitive and that have travelled from the other side of the Universe.
The term “Big Data” is often problematic, in that it is highly subjective. However, the SKA will undoubtedly be the largest Big Data project in the known Universe. The first phase of the project will generate on the order of exabytes (= 1000 petabytes) of raw data from the antennas every day. The Australian counterpart of the project alone—SKA-Low—will generate 157 terabytes of data every second. Overall, this amounts to ≈5 zettabytes ( 10 6  petabytes) of data that will be generated by the facility each year throughout the first phase of the project. This latter data rate is equivalent to 5 × the estimated global Internet traffic in 2015.1
This data flood from the SKA will have implications for medical researchers, neuroscientists, data scientists, quantitative financial analysts and the general public throughout the global community, in terms of how we handle and process such large datasets. The infrastructure for handling these data rates does not currently exist. To some extent, new technologies will enable us to process the data when the facility comes online in 2023. However, new techniques, algorithms and infrastructure also need to be developed in order to solve these data challenges. In this paper, we discuss current efforts at developing these new techniques and our contribution to various aspects of SKA development.

2. Magnetism Surveys as Prototyping Test-Beds

The science case for the SKA – a state-of-the-art radio telescope – covers a diverse range of topics, which includes challenging questions such as: How do galaxies evolve? What is the nature of dark energy? Was Einstein right about gravity? What generates giant magnetic fields in space? and even whether we are alone in the Cosmos? [1]. For the purpose of prototyping, SKA-like science pipelines could be developed to cover the technical requirements for any of these areas. We have here chosen to focus on the science case of magnetic fields, which can be advantageous from a prototyping perspective as the data processing requires full-Stokes polarization imaging in I, Q, U and V [2], together with Faraday tomography [3,4,5,6], which intrinsically has substantial computing requirements. The Big Data challenges associated with Faraday tomography will be further increased at SKA sensitivities, for which a ‘forest’ of signatures is expected to be generated along each line of sight [7].
One of the key questions for magnetic field studies is to detect magnetism on the largest cosmic scales. At such scales, the “cosmic web” (shown in Figure 2) consists of filaments of galaxies and empty desolate voids, which occupy the vast majority of the volume of the Universe. While various predictions have been made as to the strength of these intergalactic magnetic fields [8,9,10,11,12,13,14,15,16,17], the magnetic fields in this medium are so weak that they remain undetected observationally [18]. One of the best techniques for measuring cosmic magnetic fields is Faraday tomography, which can extract the Faraday rotation measure (RM) properties of extragalactic sources [19] and thereby provides a unique probe of fundamental physics. For the purpose of measuring magnetism in the cosmic web, we require large statistical samples with highly precise Faraday rotation and polarized fraction measurements. Samples with ∼40,000 sources are currently available [20], and the SKA itself is anticipated to generate up to 15 million measurements [7]. One way to explore an intermediate number of sources is by using currently available SKA pathfinder and precursor telescopes, of which one example is the LOw Frequency ARray (LOFAR; [21]).
LOFAR is a huge data producer, with a raw data rate of around 10 Tbit/s. Beam-forming at the station level is able to reduce the long-range data transport rate to around ∼150 Gbit/s, although this still requires dedicated fibre networks to transport the data. As an indication of scale, this is comparable to the bandwidth requirements of the whole of Netflix: in 2015, the peak Netflix data pipeline rate was 192 Gbit/s during peak hours. Following data transport, LOFAR data are processed at the University of Groningen by an IBM Blue Gene/P supercomputer offering about 30 Tflop/s of processing power [21]. A key area at which the LOFAR telescope excels is in low-frequency radio surveys, and one survey that has been conducted is the Multifrequency Snapshot Sky Survey (MSSS; [23]). The polarized counterpart of this survey is the MSSS All-Sky Polarization Survey (MAPS), which will enable Faraday tomography of the entire LOFAR sky. This survey is currently under development by the MSSS and MAPS teams (Heald et al., this volume), in addition to Southern sky counterparts such as POGS [24]. Surveys such as MAPS can be useful test-beds for science pipelines with the SKA and for estimating the imaging performance of a spectral imaging pipeline, which may potentially constitute one core component of the data processing requirements (e.g., [25]). We have therefore been working alongside the MSSS/MAPS teams in developing a parallelised SKA spectral imaging pipeline, which can be tested on LOFAR data.
MSSS/MAPS constitutes 50 terabytes of archived visibility data in measurement set format. The intention is to image the survey at 45 arcsec angular resolution. The survey has the goal of providing a direct detection of magnetic fields in the “cosmic web” of galaxy filaments and voids (e.g., [26]), of producing an “RM grid” at 150 MHz (e.g., [27]) and also of furthering our understanding of extragalactic radio sources (e.g., [28,29]), with progress having recently been made in charting out the evolution of cosmic magnetic fields in galaxies, with intervening galaxy experiments now spanning a redshift range from z = 0.005 [30] to z = 2 [18]. MAPS will be able to provide up to ≈200,000 sources for such studies, with a conservative estimate of positive RM detections towards 100–200 sources due to strong Faraday depolarization effects at long wavelengths. The resulting catalogue of RMs, polarized fractions and observational limits can be used to constrain the intergalactic magnetic field. This is only possible due to the high RM precision available with LOFAR, which enables Faraday tomography with a rotation measure spread function (RMSF)—the point spread function in Faraday space—of FWHM 1.3  rad m 2 [31,32,33,34,35,36,37,38,39]. As a bonus outcome of our science pipeline prototyping work, in addition to testing the deployment and operation of science for SKA, this project will also help to enable tight constraints on the magnetised large-scale structure of the Universe. The SKA therefore provides unique opportunities to contribute to the technical development of a scientific instrument that has considerable economic potential across numerous sectors, while simultaneously contributing to and enabling modern scientific discoveries by processing data from pre-existing pathfinder and precursor telescopes.
As a serialised pipeline, it had previously been estimated that MAPS would require up to two years of CPU time in order to just produce Stokes Q and U images. However, due to the parallelisable and scalable computing infrastructure now available, the survey can now be completed on a feasible time scale. The scalability is primarily related to Dask itself, which has been tested on clusters containing thousands of cores. This same infrastructure could also be applied to other magnetism surveys, including for example, data from the Australia Square Kilometre Array Pathfinder (ASKAP) [40] and the Karl G. Jansky Very Large Array (VLA) [41].

3. An SKA-Like Science Pipeline

Faraday tomography studies of magnetic fields require every single channel across the observing bandwidth to be imaged and CLEANed individually. In the case of SKA, this will require 65,000 separate images in all Stokes parameters. This process lends itself to parallelism in a trivial manner, as it is possible to image each channel simultaneously on multiple cores distributed across a cluster in a high performance computing (HPC) environment.
One plausible method for parallelisation of this process is the Python-based Dask 2. Dask is open-source and freely available, with schedulers that scale to thousand-node clusters. Dask can, in some circumstances, bypass the Global Interpreter Lock (GIL) when using pure Python objects and numeric code such as NumPy or Scikit-Learn. In particular, the Dask distributed library enables a Python function to be submitted across a cluster for task-based parallelism, generating a number of futures that can be monitored, controlled and computed as needed. The flexibility of Dask enables parallelism to be implemented in just a few lines of code. For example, the following code snippet sets up a Dask client, submits jobs across the cluster, and then monitors the progress of the running futures. The Q U imaging function could, in principle, call any imaging code of interest to the user.
import daskfrom dask.distributed import Clientfrom distributed.diagnostics import progressclient = Client(’localhost:8786’)futures = [client.submit(qu_imaging, data[chan]) for chan in range(65000)]
progress(futures) # show progress bar
def qu_imaging(data):
      # Run imaging code/software of choice.
Dask therefore represents one particular scalable method that we would like to further investigate for SKA-like science pipelines. The currently implemented science pipeline is purely Pythonic, and we have written our own interface using “Dask distributed”, which submits and monitors the running imaging processes. The completed code will be open-source and available via GitHub, so that it is possible to download the code for application to one’s favourite telescope of choice.
The science pipeline implements a suite of algorithms that are particularly specialised for the SKA. These include calibration of ionospheric Faraday rotation—which has been previously studied for LOFAR (e.g., [42])—and can be corrected by using a total electron content model [43]. The pipeline also enables a complex CLEAN deconvolution for polarimetric data [44], the generation of Faraday moment images to enable highly-complete source-finding [45] and, finally, the application of Faraday tomography [46], together with a complex cross-correlation-based RM-CLEAN [47]. GPUs have already been widely deployed in the search for transients [48], and as part of our pipeline development work, we have now also begun considering the capacity for deploying Faraday tomography algorithms on GPUs in a manner that is scalable for SKA.
An initial parallelised version of this SKA-like science pipeline is now being tested. Preliminary results indicate that the pipeline significantly reduced the time required to process a single pointing of the sky. For a test LOFAR dataset, a serialised code took 11 h to complete, whereas a parallelised Dask pipeline required just 8 min. In addition, source parameterisation measurements (sky coordinates and flux density) of a pulsar detected using these test datasets have been found to be consistent with those from other radio astronomy packages, thereby confirming the technical accuracy and scientific veracity of the science pipeline.

4. Conclusions

Our development work will test the scalability and the implementation of SKA science pipelines. We are aiming to test a fully working and deployable spectral imaging and Faraday tomography pipeline by the end of 2018.
Our prototyping work also has multiple additional outcomes. It allows us to optimise and validate the scalability of the Faraday tomography algorithm in the SKA-era. It allows us to provide a realistic SKA-like data pipeline, which can be optimised for, deployed and tested on HPC infrastructure. It also provides bonus outcomes: in terms of generating all-sky datasets that can be provided for science publications and the potential discovery of the magnetised large-scale structure of the Universe.

Author Contributions

Conceptualisation, J.F., B.M. and W.A. Methodology, J.F., B.M., F.D., S.S. and W.A. Software, J.F., B.M., F.D., S.S. and W.A. Validation, J.F., B.M. and F.D. Formal analysis, J.F. Writing, original draft preparation, J.F. Writing, review and editing, J.F. Visualisation, J.F. Supervision, B.M. and W.A. Project administration, W.A. Funding acquisition, W.A. Authorship is limited to those who have contributed substantially to the work reported.


This work is supported by STFC Grants: SKA Preconstruction funding supplement (ST/N003713/1), SKA Preconstruction update (ST/P005446/1) and SKA Preconstruction 2017–18 (ST/R000557/1).


We are grateful to the wider Scientific Computing group at OeRC, including Karel Adámek, Anna Brown and Jan Novotny, for support and insightful comments that helped to improve this document. We are also grateful to the MSSS/MAPS teams, led by George Heald and Jess Broderick, for allowing us to test these pipelines with LOFAR data. LOFAR, designed and constructed by ASTRON, has facilities in several countries, which are owned by various parties (each with their own funding sources) and which are collectively operated by the International LOFAR Telescope (ILT) foundation under a joint scientific policy.

Conflicts of Interest

The authors declare no conflict of interest.


The following abbreviations are used in this manuscript:
ASKAPAustralia SKA Pathfinder
CPUcentral processing unit
GILGlobal Interpreter Lock
GPUgraphics processing unit
HPChigh performance computing
LOFARLOw Frequency ARray
MAPSMSSS All-Sky Polarization Survey
MSSSMultifrequency Snapshot Sky Survey
POGSPOlarised GLEAM Survey
RFIradio-frequency interference
RMrotation measure
RMSFrotation measure spread function
SKASquare Kilometre Array
STFCScience and Technology Facilities Council
VLAKarl G. Jansky Very Large Array


  1. Braun, R.; Bourke, T.; Green, J.A.; Keane, E.; Wagg, J. Advancing Astrophysics with the Square Kilometre Array. In Proceedings of the Advancing Astrophysics with the Square Kilometre Array (AASKA14), Giardini Naxos, Italy, 9–13 June 2014. [Google Scholar]
  2. Hamaker, J.P.; Bregman, J.D.; Sault, R.J. Understanding radio polarimetry. I. Mathematical foundations. Astron. Astrophys. Suppl. Ser. 1996, 117, 137–147. [Google Scholar] [CrossRef][Green Version]
  3. Akahori, T.; Fujita, Y.; Ichiki, K.; Ideguchi, S.; Kudoh, T.; Kudoh, Y.; Machida, M.; Nakanishi, H.; Ohno, H.; Ozawa, T.; et al. Resolving 4-D Nature of Magnetism with Depolarization and Faraday Tomography: Japanese SKA Cosmic Magnetism Science. arXiv, 2016; arXiv:1603.01974. [Google Scholar]
  4. Akahori, T.; Nakanishi, H.; Sofue, Y.; Fujita, Y.; Ichiki, K.; Ideguchi, S.; Kameya, O.; Kudoh, T.; Kudoh, Y.; Machida, M.; et al. Cosmic magnetism in centimeter- and meter-wavelength radio astronomy. Publ. Astron. Soc. Jpn. 2018, 70. [Google Scholar] [CrossRef]
  5. Burn, B.J. On the depolarization of discrete radio sources by Faraday dispersion. Mon. Not. R. Astron. Soc. 1966, 133, 67. [Google Scholar] [CrossRef]
  6. Kumazaki, K.; Akahori, T.; Ideguchi, S.; Kurayama, T.; Takahashi, K. Properties of intrinsic polarization angle ambiguities in Faraday tomography. Publ. Astron. Soc. Jpn. 2014, 66, 61. [Google Scholar] [CrossRef]
  7. Gaensler, B.; Agudo, I.; Akahori, T.; Banfield, J.; Beck, R.; Carretti, E.; Farnes, J.; Haverkorn, M.; Heald, G.; Jones, D.; et al. Broadband Polarimetry with the Square Kilometre Array: A Unique Astrophysical Probe. In Proceedings of the Advancing Astrophysics with the Square Kilometre Array (AASKA14), Giardini Naxos, Italy, 9–13 June 2014. [Google Scholar]
  8. Akahori, T.; Gaensler, B.M.; Ryu, D. Statistical Techniques for Detecting the Intergalactic Magnetic Field from Large Samples of Extragalactic Faraday Rotation Data. Astrophys. J. 2014, 790, 123. [Google Scholar] [CrossRef]
  9. Akahori, T.; Kumazaki, K.; Takahashi, K.; Ryu, D. Exploring the intergalactic magnetic field by means of Faraday tomography. Publ. Astron. Soc. Jpn. 2014, 66, 65. [Google Scholar] [CrossRef]
  10. Akahori, T.; Ryu, D. RM due to magnetic fields in the cosmic web and SKA observations. Highlights Astron. 2015, 16, 407. [Google Scholar] [CrossRef]
  11. Akahori, T.; Ryu, D. Faraday Rotation Measure due to the Intergalactic Magnetic Field. II. The Cosmological Contribution. Astrophys. J. 2011, 738, 134. [Google Scholar] [CrossRef]
  12. Akahori, T.; Ryu, D. Faraday Rotation Measure Due to the Intergalactic Magnetic Field. Astrophys. J. 2010, 723, 476. [Google Scholar] [CrossRef]
  13. Akahori, T.; Ryu, D.; Gaensler, B.M. Fast Radio Bursts as Probes of Magnetic Fields in the Intergalactic Medium. Astrophys. J. 2016, 824, 105. [Google Scholar] [CrossRef]
  14. Akahori, T.; Ryu, D.; Kim, J.; Gaensler, B.M. Simulated Faraday Rotation Measures toward High Galactic Latitudes. Astrophys. J. 2013, 767, 150. [Google Scholar] [CrossRef]
  15. Vazza, F.; Ferrari, C.; Brüggen, M.; Bonafede, A.; Gheller, C.; Wang, P. Forecasts for the detection of the magnetised cosmic web from cosmological simulations. Astron. Astrophys. 2015, 580, A119. [Google Scholar] [CrossRef][Green Version]
  16. Vazza, F.; Brueggen, M.; Gheller, C.; Ferrari, C.; Bonafede, A. Detecting the cosmic web with radio surveys. In Proceedings of the Many Facets of Extragalactic Radio Surveys: Towards New Scientific Challenges, Bologna, Italy, 20–23 October 2015. [Google Scholar]
  17. Vazza, F.; Brüggen, M.; Hinz, P.M.; Wittor, D.; Locatelli, N.; Gheller, C. Probing the origin of extragalactic magnetic fields with Fast Radio Bursts. Mon. Not. R. Astron. Soc. 2018, 480, 3907–3915. [Google Scholar] [CrossRef]
  18. Farnes, J.S.; Rudnick, L.; Gaensler, B.M.; Haverkorn, M.; O’Sullivan, S.P.; Curran, S.J. Observed Faraday Effects in Damped Lyα Absorbers and Lyman Limit Systems: The Magnetized Environment of Galactic Building Blocks at Redshift = 2. Astrophys. J. 2017, 841, 67. [Google Scholar] [CrossRef]
  19. Farnes, J.S.; Gaensler, B.M.; Carretti, E. A Broadband Polarization Catalog of Extragalactic Radio Sources. Astrophys. J. Suppl. Ser. 2014, 212, 15. [Google Scholar] [CrossRef]
  20. Taylor, A.R.; Stil, J.M.; Sunstrum, C. A Rotation Measure Image of the Sky. Astrophys. J. 2009, 702, 1230–1236. [Google Scholar] [CrossRef]
  21. Van Haarlem, M.P.; Wise, M.W.; Gunst, A.W.; Heald, G.; McKean, J.P.; Hessels, J.W.T.; de Bruyn, A.G.; Nijboer, R.; Swinbank, J.; Fallows, R.; et al. LOFAR: The LOw-Frequency ARray. Astron. Astrophys. 2013, 556, A2. [Google Scholar] [CrossRef][Green Version]
  22. Springel, V.; White, S.D.M.; Jenkins, A.; Frenk, C.S.; Yoshida, N.; Gao, L.; Navarro, J.; Thacker, R.; Croton, D.; Helly, J.; et al. Simulations of the formation, evolution and clustering of galaxies and quasars. Nature 2005, 435, 629–636. [Google Scholar] [CrossRef] [PubMed][Green Version]
  23. Heald, G.H.; Pizzo, R.F.; Orrú, E.; Breton, R.P.; Carbone, D.; Ferrari, C.; Hardcastle, M.J.; Jurusik, W.; Macario, G.; Mulcahy, D.; et al. The LOFAR Multifrequency Snapshot Sky Survey (MSSS). I. Survey description and first results. Astron. Astrophys. 2015, 582, A123. [Google Scholar] [CrossRef]
  24. Riseley, C.J.; Lenc, E.; Van Eck, C.L.; Heald, G.; Gaensler, B.M.; Anderson, C.S.; Hancock, P.J.; Hurley-Walker, N.; Sridhar, S.S.; White, S.V. The POlarised GLEAM Survey (POGS) I: First Results from a Low-Frequency Radio Linear Polarisation Survey of the Southern Sky. Publ. Astron. Soc. Aust. 2018, in press. [Google Scholar]
  25. Mort, B.; Dulwich, F.; Razavi-Ghods, N.; de Lera Acedo, E.; Grainge, K. Analysing the impact of far-out sidelobes on the imaging performance of the SKA-LOW telescope. Mon. Not. R. Astron. Soc. 2017, 465, 3680–3692. [Google Scholar] [CrossRef]
  26. Bray, J.D.; Scaife, A.M.M. An upper limit on the strength of the extragalactic magnetic field from ultra-high-energy cosmic-ray anisotropy. Astrophys. J. 2018, 861, 3. [Google Scholar] [CrossRef]
  27. Van Eck, C.L.; Haverkorn, M.; Alves, M.I.R.; Beck, R.; Best, P.; Carretti, E.; Chyży, K.T.; Farnes, J.S.; Ferrière, K.; Hardcastle, M.J.; et al. Polarized point sources in the LOFAR Two-meter Sky Survey: A preliminary catalog. Astron. Astrophys. 2018, 613, A58. [Google Scholar] [CrossRef][Green Version]
  28. Lamee, M.; Rudnick, L.; Farnes, J.S.; Carretti, E.; Gaensler, B.M.; Haverkorn, M.; Poppi, S. Magnetic Field Disorder and Faraday Effects on the Polarization of Extragalactic Radio Sources. Astrophys. J. 2016, 829, 5. [Google Scholar] [CrossRef]
  29. O’Sullivan, S.P.; Purcell, C.R.; Anderson, C.S.; Farnes, J.S.; Sun, X.H.; Gaensler, B.M. Broad-band, radio spectro-polarimetric study of 100 radiative-mode and jet-mode AGN. Mon. Not. R. Astron. Soc. 2017, 469, 4034–4062. [Google Scholar] [CrossRef]
  30. Gupta, N.; Srianand, R.; Farnes, J.S.; Pidopryhora, Y.; Vivek, M.; Paragi, Z.; Noterdaeme, P.; Oosterloo, T.; Petitjean, P. Revealing H I gas in emission and absorption on pc to kpc scales in a galaxy at z∼0.017. Mon. Not. R. Astron. Soc. 2018, 476, 2432. [Google Scholar] [CrossRef]
  31. Clarke, A.O.; Heald, G.; Jarrett, T.; Bray, J.D.; Hardcastle, M.J.; Cantwell, T.M.; Scaife, A.M.M.; Brienza, M.; Bonafede, A.; Breton, R.P.; et al. LOFAR MSSS: Discovery of a 2.56 Mpc giant radio galaxy associated with a disturbed galaxy group. Astron. Astrophys. 2017, 601, A25. [Google Scholar] [CrossRef][Green Version]
  32. Hardcastle, M.J.; Gürkan, G.; van Weeren, R.J.; Williams, W.L.; Best, P.N.; de Gasperin, F.; Rafferty, D.A.; Read, S.C.; Sabater, J.; Shimwell, T.W.; et al. LOFAR/H-ATLAS: A deep low-frequency survey of the Herschel-ATLAS North Galactic Pole field. Mon. Not. R. Astron. Soc. 2016, 462, 1910–1936. [Google Scholar] [CrossRef]
  33. Iacobelli, M.; Haverkorn, M.; Orrú, E.; Pizzo, R.F.; Anderson, J.; Beck, R.; Bell, M.R.; Bonafede, A.; Chyzy, K.; Dettmar, R.-J.; et al. Studying Galactic interstellar turbulence through fluctuations in synchrotron emission. First LOFAR Galactic foreground detection. Astron. Astrophys. 2013, 558, A72. [Google Scholar] [CrossRef]
  34. Jelić, V.; de Bruyn, A.G.; Mevius, M.; Abdalla, F.B.; Asad, K.M.B.; Bernardi, G.; Brentjens, M.A.; Bus, S.; Chapman, E.; Ciardi, B.; et al. Initial LOFAR observations of epoch of reionization windows. II. Diffuse polarized emission in the ELAIS-N1 field. Astron. Astrophys. 2014, 568, A101. [Google Scholar] [CrossRef]
  35. Moldón, J.; Deller, A.T.; Wucknitz, O.; Jackson, N.; Drabent, A.; Carozzi, T.; Conway, J.; Kapińska, A.D.; McKean, J.P.; Morabito, L.; et al. The LOFAR long baseline snapshot calibrator survey. Astron. Astrophys. 2015, 574, A73. [Google Scholar] [CrossRef][Green Version]
  36. Mulcahy, D.D.; Horneffer, A.; Beck, R.; Heald, G.; Fletcher, A.; Scaife, A.; Adebahr, B.; Anderson, J.M.; Bonafede, A.; Brüggen, M.; et al. The nature of the low-frequency emission of M 51. First observations of a nearby galaxy with LOFAR. Astron. Astrophys. 2014, 568, A74. [Google Scholar] [CrossRef]
  37. Sotomayor-Beltran, C.; Sobey, C.; Hessels, J.W.T.; de Bruyn, G.; Noutsos, A.; et al. Calibrating high-precision Faraday rotation measurements for LOFAR and the next generation of low-frequency radio telescopes. Astron. Astrophys. 2013, 552, A58. [Google Scholar] [CrossRef][Green Version]
  38. Van Eck, C.L.; Haverkorn, M.; Alves, M.I.R.; Beck, R.; de Bruyn, A.G.; Enßlin, T.; Farnes, J.S.; Ferrière, K.; Heald, G.; Horellou, C.; et al. Faraday tomography of the local interstellar medium with LOFAR: Galactic foregrounds towards IC 342. Astron. Astrophys. 2017, 597, A98. [Google Scholar] [CrossRef][Green Version]
  39. Varenius, E.; Conway, J.E.; Martí-Vidal, I.; Beswick, R.; Deller, A.T.; Wucknitz, O.; Jackson, N.; Adebahr, B.; Pérez-Torres, M.A.; Chyży, K.T.; et al. Subarcsecond international LOFAR radio images of the M82 nucleus at 118 MHz and 154 MHz. Astron. Astrophys. 2015, 574, A114. [Google Scholar] [CrossRef][Green Version]
  40. Johnston, S.; Bailes, M.; Bartel, N.; Baugh, C.; Bietenholz, M.; Blake, C.; Braun, R.; Brown, J.; Chatterjee, S.; Darling, J.; et al. Science with the Australian Square Kilometre Array Pathfinder. Publ. Astron. Soc. Aust. 2007, 24, 174. [Google Scholar] [CrossRef]
  41. Lacy, M.; Baum, S.A.; Chandler, C.J.; Chatterjee, S.; Murphy, E.J.; Myers, S.T.; VLASS Survey Science Group. The VLA Sky Survey (VLASS): Description and Science Goals; American Astronomical Society: Washington, DC, USA, 2016. [Google Scholar]
  42. Mevius, M.; van der Tol, S.; Pandey, V.N.; Vedantham, H.K.; Brentjens, M.A.; de Bruyn, A.G.; Abdalla, F.B.; Asad, K.M.B.; Bregman, J.D.; Brouw, W.N.; Bus, S.; et al. Probing ionospheric structures using the LOFAR radio telescope. Radio Sci. 2016, 51, 927–941. [Google Scholar] [CrossRef][Green Version]
  43. Mevius, M. RMextract: Ionospheric Faraday Rotation Calculator, Astrophysics Source Code Library, 2018, (ascl: 1806.024). Available online: (accessed on 19 November 2018).
  44. Pratley, L.; Johnston-Hollitt, M. An improved method for polarimetric image restoration in interferometry. Mon. Not. R. Astron. Soc. 2016, 462, 3483. [Google Scholar] [CrossRef]
  45. Farnes, J.S.; Heald, G.; Junklewitz, H.; Mulcahy, D.D.; Haverkorn, M.; Van Eck, C.L.; Riseley, C.J.; Brentjens, M.; Horellou, C.; Vacca, V.; et al. Source finding in linear polarization for LOFAR, and SKA predecessor surveys, using Faraday moments. Mon. Not. R. Astron. Soc. 2018, 474, 3280. [Google Scholar] [CrossRef]
  46. Brentjens, M.A.; de Bruyn, A.G. Faraday rotation measure synthesis. Astron. Astrophys. 2005, 441, 1217–1228. [Google Scholar] [CrossRef][Green Version]
  47. Heald, G. The Faraday rotation measure synthesis technique. Cosm. Magn. Fields From Planets Stars Galaxies 2009, 259, 591–602. [Google Scholar] [CrossRef]
  48. Armour, W.; Karastergiou, A.; Giles, M.; Williams, C.; Magro, A.; Zagkouris, K.; Roberts, S.; Salvini, S.; Dulwich, F.; Mort, B. A GPU-based Survey for Millisecond Radio Transients Using ARTEMIS. In Proceedings of the Astronomical Data Analysis Software and Systems XXI, Paris, France, 6–10 November 2011; Ballester, P., Egret, D., Lorente, N.P.F., Eds.; Astronomical Society of the Pacific: San Francisco, CA, USA, 2012; p. 33. [Google Scholar]
As estimated by the Cisco Systems Visual Networking Index (VNI).
Figure 1. (Top) The dishes for SKA-Mid, located in South Africa, will generate 62 exabytes of raw data per year, which is enough to fill 340,000 average laptops with content every day. (Bottom) The aperture arrays for SKA-Low, located in Australia, will generate five zettabytes of data per year. Unlike conventional dishes, the “beam” of these antennas is formed electronically, allowing each station to look at multiple regions of the sky simultaneously. The total field of view is essentially limited by the signal processing capacity. Image credits: SKA Organisation.
Figure 1. (Top) The dishes for SKA-Mid, located in South Africa, will generate 62 exabytes of raw data per year, which is enough to fill 340,000 average laptops with content every day. (Bottom) The aperture arrays for SKA-Low, located in Australia, will generate five zettabytes of data per year. Unlike conventional dishes, the “beam” of these antennas is formed electronically, allowing each station to look at multiple regions of the sky simultaneously. The total field of view is essentially limited by the signal processing capacity. Image credits: SKA Organisation.
Galaxies 06 00120 g001
Figure 2. The structure of the intergalactic medium can be illustrated by the Millennium simulation [22]. This “cosmic web” consists of filaments of galaxies and empty desolate voids, which occupy the vast majority of the volume of the Universe. Measuring the elusive cosmic magnetic fields in this diffuse plasma is fundamentally dependent on the computationally-expensive technique of Faraday tomography, which can have similar computational overheads to de-dispersion for observations of transients.
Figure 2. The structure of the intergalactic medium can be illustrated by the Millennium simulation [22]. This “cosmic web” consists of filaments of galaxies and empty desolate voids, which occupy the vast majority of the volume of the Universe. Measuring the elusive cosmic magnetic fields in this diffuse plasma is fundamentally dependent on the computationally-expensive technique of Faraday tomography, which can have similar computational overheads to de-dispersion for observations of transients.
Galaxies 06 00120 g002

Share and Cite

MDPI and ACS Style

Farnes, J.; Mort, B.; Dulwich, F.; Salvini, S.; Armour, W. Science Pipelines for the Square Kilometre Array. Galaxies 2018, 6, 120.

AMA Style

Farnes J, Mort B, Dulwich F, Salvini S, Armour W. Science Pipelines for the Square Kilometre Array. Galaxies. 2018; 6(4):120.

Chicago/Turabian Style

Farnes, Jamie, Ben Mort, Fred Dulwich, Stef Salvini, and Wes Armour. 2018. "Science Pipelines for the Square Kilometre Array" Galaxies 6, no. 4: 120.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop