An introduction to scientific programming with
http://stevenbamford.com/pythoncourse
Steven Bamford
•
•
•
Part 1: Introduction
• • •
Why (and why not) to use a modern, high-level, scripting language Why Python is awesome Introduction to the language
• •
Good programming practice versus 'thinking aloud’ Python 2.x versus 3.x
•
startup, syntax, constructs, functions, classes, getting help
Part 2: Scientific programming in Python
• • • •
Arrays – Numerical Python Using arrays wisely Plotting Scientific Python
•
Other tools
•
An example analysis program
•
optimisation, interpolation, statistics, filtering, integration, …
•
GNU Scientific Library, R, …
Part 3: Python for observers
• •
Handling FITS files PyRAF – scripting IRAF with Python
An introduction to scientific programming with
Part 1: Introduction
•
Modern scripting languages:
• • •
Python, Perl, Ruby, IDL, … High-level Interactive interpreter
• •
Ease of use
• • •
Encourages scripting, rather than one-off analysis
Speed of development
Permanent record Repeatability
• • •
If you want fastest possible performance Highly parallel code Need low-level control
• • • • • • • • • •
Designed to be easy to learn and use – clear syntax Well documented Powerful, flexible, fully-featured programming language ‘Batteries included’ Comprehensive scientific tools Fast Interpreter, introspection Runs everywhere Completely free You already have it
• • • •
Less stress Get more science done Widely used and growing popularity Throughout academia and industry
• •
• •
NASA, AstraZeneca, Google, Industrial Light & Magic, Philips,… Web services, engineering, science, air traffic control, quantitative finance, games, education, data management, …
Python programmers in demand Easy introduction to general programming concepts
Why not?
•
Existing code for your project in another language, but still…
http://python.org
>>> 2+2 4 >>> # This is a comment ... 2+2 4 >>> 2+2.0 # and a comment on the same line as code 4.0 >>> (50-5*6)/4 5 >>> width = 20 # assignment, no type declaration >>> height = 5*9 >>> width * height 900 >>> x = y = z = 0 # zero x, y and z >>> y 0 >>> n Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'n' is not defined
2+2 # This is a comment 2+2 2+2.0 # and a comment on the same line as code (50-5*6)/4 width = 20 # assignment, no type declaration height = 5*9 width * height x = y = z = 0 # zero x, y and z y
• •
Can write in a text editor and copy and paste into interpreter
•
Can save and use interactively in future sessions (import)
Can save and execute from command line: $ python test.py
>>> 'spam and eggs' 'spam and eggs' >>> 'doesn\'t' "doesn't" >>> "doesn't" "doesn't" >>> '"Yes," he said.' '"Yes," he said.’ >>> hello = ‘Greetings!’ >>> hello ‘Greetings!’ >>> print(hello) Greetings! >>> print(hello + ‘ How do you do?’) Greetings! How do you do? >>> print(hello, ‘How do you do?’) Greetings! How do you do? >>> howdo = ’How do you do?’ >>> print(hello+’ ‘+howdo) Greetings! How do you do?
>>> 10 + 3 13 >>> 10 - 3 7 >>> 10 * 3 30 >>> 10 / 3 3 >>> 10 // 3 3 >>> 10 % 3 1 >>> 10**3 1000 >>> 10 + 3 * 5 # *,/ then +,25 >>> (10 + 3) * 5 65 >>> -1**2 # -(1**2) -1
>>> 10.0 + 3.0 13.0 >>> 10.0 - 3.0 7.0 >>> 10.0 * 3 30.0 >>> 10.0 / 3 3.3333333333333335 >>> 10.0 // 3 3.0 >>> 10.0 % 3.0 1.0 >>> 10.0**3 1000.0 >>> 4.2 + 3.14 7.3399999999999999 >>> 4.2 * 3.14 13.188000000000001
Augmented assignment: >>> >>> >>> 28 >>> >>> 3.5
a = 20 a += 8 a
Functions:
a /= 8.0 a
Comparisons: >>> 5 * 2 == 4 + 6 True >>> 0.12 * 2 == 0.1 + 0.14 False >>> a = 0.12 * 2; b = 0.1 + 0.14 >>> eps = 0.0001 >>> a - eps < b < a + eps True
>>> abs(-5.2) 5.2 >>> sqrt(25) 5.0
Lists: >>> a = [1, 2, 4, 8, 16] # list of ints >>> c = [4, ‘candles’, 4.0, ‘handles’] # can mix types >>> c[1] ‘candles’ >>> c[2] = ‘knife’ >>> c[-1] # negative indices count from end ‘handles’ >>> c[1:3] # slicing
[‘candles’, ‘knife’] >>> c[2:]
# omitting defaults to start or end
[‘knife’, ‘handles’] >>> c[0:4:2]
# variable stride (could just write c[::2])
[4, ‘knife’] >>> a + c # concatenate [1, 2, 4, 8, 16, 4, ‘candles’, ‘knife’, ‘handles’] >>> len(a) 5
Tuples: >>> q = (1, 2, 4, 8, 16) # tuple of ints >>> r = (4, ‘candles’, 4.0, ‘handles’) # can mix types >>> s = (‘lonely’,) # singleton >>> t = () # empty >>> r[1] ‘candles’ >>> r[2] = ‘knife’ # cannot change tuples Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment >>> u = 3, 2, 1
# parentheses not necessary
>>> v, w = ‘this’, ‘that >>> v ‘this’ >>> w ‘that’
Dictionaries: >>> a = {‘eyecolour’: ‘blue’, ‘height’: 152.0, 42: ‘the answer‘} >>> a[‘age’] = 28 >>> a {42: 'the answer', 'age': 28, 'eyecolour': 'blue', 'height': 152.0} >>> del(a[‘height’]) >>> a {42: 'the answer', 'age': 28, 'eyecolour': 'blue'} >>> b = {} >>> b[‘hello’] = ‘Hi!’ >>> a.keys() [42, 'age’, 'eyecolour’] >>> a.values() ['the answer’, 28, 'blue’]
>>> a = 4; b = 3 >>> if a > b: ... result = ‘bigger’ ... c = a - b ... >>> print(result, c) bigger 1 >>> a = 1; b = 3 >>> if a > b: ... result = ‘bigger’ ... elif a == b: ... result = ‘same’ ... else: # i.e. a < b ... result = ‘smaller’ ... >>> print(result) smaller >>> if a < b: print ‘ok’ ok
•
Indentation is important!
• • •
be consistent use four spaces do not use tabs
Comparison operators: == > >= is in
!= < <= is not not in
Boolean operators: and or not
>>> if ‘Steven’ in [‘Bob’, ‘Amy’, ‘Steven’, ‘Fred’]: ... print ‘Here!’ ... Here! >>> if ‘Carol’ not in [‘Bob’, ‘Amy’, ‘Steven’, ‘Fred’]: ... print ‘Away!’ ... Away! >>> test = a == b >>> if test: print ‘Equal’ ‘Equal’
>>> a = b = 0 >>> while a < 10: ... a += 3 ... print(a) ...
3 6 9 12 >>> while True: ... b += 3 ... if b >= 10: break ... print(b) 3 6 9
>>> for i in [2, 5, 3]: ... print(i**2) 4 25 9 >>> for j in range(5): 0 1 2 3 4
print(j)
>>> range(3, 10, 2) [3,5,7,9] >>> d = >>> for ... this is that is
{‘this’: 2, ‘that’: 7} k, v in d.items(): print(‘%s is %i’%(k, v)) 2 7
>>> ... ... ... ... >>> 8.0 >>> 4.0 >>> 4.0 >>> 5.0 >>> 6.0 >>> 6.0
def my_func(x, y=0.0, z=1.0): a = x + y b = a * z return b my_func(1.0, 3.0, 2.0) my_func(1.0, 3.0) my_func(1.0, y=3.0) my_func(5.0) my_func(2.0, z=3.0) my_func(x=2.0, z=3.0)
>>> >>> >>> [2, >>> 1 >>> 2 >>> >>> [6,
a = [2, 5, 3, 6, 5] a.sort() print(a) 3, 5, 5, 6] a.count(3) a.count(5) a.reverse() print(a) 5, 5, 3, 2]
>>> d = {‘black’: 100, ‘grey’: 50, ‘white’: 0} >>> d.values() [0, 50, 100] >>> s = ‘-‘.join((‘2009’, ‘07’, ‘07’)) >>> print(s) 2009-07-07
â&#x20AC;˘â&#x20AC;Ż
A neat way of creating lists (and arrays) without writing a loop
my_fav_num = [3, 17, 22, 46, 71, 8] even_squared = [] for n in my_fav_num: if n%2 == 0: even_squared.append(n**2) # in one line: even_better = [n**2 for n in my_fav_num if n%2 == 0] # both produce [484, 2116, 64] freshfruit = ['
banana', '
loganberry ', 'passion fruit
stripped = [weapon.strip() for weapon in freshfruit] print(stripped) ['banana', 'loganberry', 'passion fruit']
']
>>> class MyClass: ... def __init__(self, x, y=1.0) ... self.my_x = x ... self.my_y = y ...
... ... ... >>> >>> 8 >>> >>> 5 >>> 8
def product(): return x*y m = MyClass(2, 4) m.product() p = MyClass(5) p.product() m.product()
• • •
Modules can contain any code Classes, functions, definitions, immediately executed code Can be imported in own namespace, or into the global namespace
>>> import math >>> math.cos(math.pi) -1.0 >>> math.cos(pi) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'pi' is not defined >>> from math import cos, pi >>> cos(pi) -1.0 >>> from math import *
>>> # My totally wicked function >>> def my_func(x, y=0.0, z=1.0): ... ”””This does some stuff. ... For example: >>> my_func(1.0, 3.0, 2.0) 8.0 Yep, it’s that good! ””” ... a = x + y ... b = a * z ... return b ...
• •
Comments before function, class, etc. are used to generate help “Docstrings”
• •
preferred way of documenting code can contain examples, which are automatically turned into tests! • See doctest module
>>> fname = ‘myfile.dat’ >>> f = file(fname) >>> lines = f.readlines() >>> f.close() >>> f = file(fname) >>> firstline = f.readline() >>> secondline = f.readline() >>> f = file(fname) >>> for l in f: >>> print l.split()[1] >>> f.close() >>> >>> >>> >>>
outfname = ‘myoutput’ outf = file(outfname, ‘w’) # second argument denotes writable outf.write(‘My very own file\n’) outf.close()
>>> sigma = 6.76/2.354 >>> print(‘sigma is %5.3f metres’%sigma) sigma is 2.872 metres >>> d = {‘bob’: 1.87, ‘fred’: 1.768} >>> for name, height in d.items(): ... print(‘%s is %.2f metres tall’%(name.capitalize(), height)) ... Bob is 1.87 metres tall Fred is 1.77 metres tall >>> >>> >>> >>> ... ... >>>
•
nsweets = range(100) calories = [i * 2.345 for i in nsweets] fout = file(‘sweetinfo.txt’, ‘w’) for i in range(nsweets): fout.write(‘%5i %8.3f\n’%(nsweets[i], calories[i])) fout.close()
Also template system
• •
Powerful help tools Every object, function, module, … can be inspected
>>> help(math) >>> help(math.cos) >>> a = [1, 2, 3] >>> help(a)
• •
New 3.x branch is intentionally backwards incompatible
• • •
print a ➔ print(a)
•
Version 2.6 contains most backward compatible changes, and can warn of usage (-3 switch) which would be an error in 3.x
• • •
2to3 (semi-)automated code translator
Various improvements, removal of obsolete code, but annoying!
1/3 == 1//3 == 0 ➔ 1/3 == 1.0/3.0 == 1.333…, 1//3 == 0 … various others …
Many useful modules not compatible with 3.x yet Use 2.6.x for now…
•
Compromise between:
• • • •
Comment clearly
•
Science, not software development
•
producing results quickly and • easy reusability and adaptation of code • code that can be quickly understood by others
Use functions Use modules Consider ‘refactoring’ before code gets too messy
http://www.python.org/doc/ (note version numbers)
An introduction to scientific programming with
Part 2: Scientific programming in Python
An introduction to scientific programming with
Course webpage: http://stevenbamford.com/teaching/python_february_2010/
•
Extra features required:
• fast, multidimensional arrays • libraries of reliable, tested scientific functions • plotting tools
•
Lists ok for storing small amounts of one-dimensional data
>>> >>> [5, >>> >>> [1, >>> [6,
• • • •
a = [1,3,5,7,9] print(a[2:4]) 7] b = [[1, 3, 5, 7, 9], [2, 4, 6, 8, 10]] print(b[0]) 3, 5, 7, 9] print(b[1][2:4]) 8]
But, can’t use directly with arithmetical operators (+, -, *, /, …) Need efficient arrays with arithmetic and better multidimensional tools Numpy
>>> import numpy
Similar to lists, but much more capable, except fixed size
>>> import numpy >>> l = [[1, 2, 3], [3, 6, 9], [2, 4, 6]] # create a list >>> a = numpy.array(l) # convert a list to an array >>> print(a) [[1 2 3] [3 6 9] [2 4 6]] >>> a.shape (3, 3) >>> print(a.dtype) # get type of an array int32 >>> print(a[0]) # this is just like a list of lists [1 2 3] >>> print(a[1, 2]) # arrays can be given comma separated indices 9 >>> print(a[1, 1:3]) # and slices [6 9] >>> print(a[:,1]) [2 6 4]
>>> >>> [[1 [3 [2 >>> >>> [[0 [9 [8
a[1, 2] = 7 print(a) 2 3] 6 7] 4 6]] a[:, 0] = [0, 9, 8] print(a) 2 3] 6 7] 4 6]]
>>> b = numpy.zeros(5) >>> print(b) [ 0. 0. 0. 0. 0.] >>> b.dtype dtype(‘float64’) >>> n = 1000 >>> my_int_array = numpy.zeros(n, dtype=numpy.int) >>> my_int_array.dtype dtype(‘int32’)
>>> c = numpy.ones(4) >>> print(c) [ 1. 1. 1. 1. ] >>> d = numpy.arange(5) >>> print(d) [0 1 2 3 4]
# just like range()
>>> d[1] = 9.7 >>> print(d) # arrays keep their type even if elements changed [0 9 2 3 4] >>> print(d*0.4) # operations create a new array, with new type [ 0. 3.6 0.8 1.2 1.6] >>> d = numpy.arange(5, dtype=numpy.float) >>> print(d) [ 0. 1. 2. 3. 4.] >>> numpy.arange(3, 7, 0.5) # arbitrary start, stop and step array([ 3. , 3.5, 4. , 4.5, 5. , 5.5, 6. , 6.5])
>>> a = numpy.arange(4.0) >>> b = a * 23.4 >>> c = b/(a+1) >>> c += 10 >>> print c [ 10. 21.7 25.6 27.55] >>> arr = numpy.arange(100, 200) >>> select = [5, 25, 50, 75, -5] >>> print(arr[select]) # can use integer lists as indices [105, 125, 150, 175, 195] >>> arr = numpy.arange(10, 20 ) >>> div_by_3 = arr%3 == 0 # comparison produces boolean array >>> print(div_by_3) [ False False True False False True False False True False] >>> print(arr[div_by_3]) # can use boolean lists as indices [12 15 18]
>>> arr.sum() 45 >>> arr.mean() 4.5 >>> arr.std() 2.8722813232690143 >>> arr.max() 9 >>> arr.min() 0 >>> div_by_3.all() False >>> div_by_3.any() True >>> div_by_3.sum() 3 >>> div_by_3.nonzero() (array([2, 5, 8]),)
>>> arr = numpy.array([4.5, 2.3, 6.7, 1.2, 1.8, 5.5]) >>> arr.sort() # acts on array itself >>> print(arr) [ 1.2 1.8 2.3 4.5 5.5 6.7] >>> x = numpy.array([4.5, >>> y = numpy.array([1.5, >>> numpy.sort(x) array([ 1.2, 1.8, 2.3, >>> print(x) [ 4.5 2.3 6.7 1.2 1.8 >>> s = x.argsort() >>> s array([3, 4, 1, 0, 5, 2]) >>> x[s] array([ 1.2, 1.8, 2.3, >>> y[s] array([ 6.2, 7.8, 2.3,
2.3, 6.7, 1.2, 1.8, 5.5]) 2.3, 4.7, 6.2, 7.8, 8.5]) 4.5,
5.5,
6.7])
4.5,
5.5,
6.7])
1.5,
8.5,
4.7])
5.5]
•
Most array methods have equivalent functions
>>> arr.sum() 45 >>> numpy.sum(arr) 45
•
Ufuncs provide many element-by-element math, trig., etc. operations
•
•
e.g., add(x1, x2), absolute(x), log10(x), sin(x), logical_and(x1, x2)
See http://numpy.scipy.org
• • • • •
Array operations are implemented in C or Fortran Optimised algorithms - i.e. fast! Python loops (i.e. for i in a:…) are much slower Prefer array operations over loops, especially when speed important Also produces shorter code, often more readable
• •
User friendly, but powerful, plotting capabilites for python
•
Once installed, to use type:
http://matplotlib.sourceforge.net/
>>> import pylab
•
Settings can be customised by editing ~/.matplotlib/matplotlibrc
•
Helpful website
• •
default font, colours, layout, etc. many examples
•
Suite of numerical and scientific tools for Python
•
http://scipy.org/
Example modules:
•
Optimisation / minimisation
•
•
Integration
•
•
http://docs.scipy.org/doc/scipy/reference/interpolate.html
Statistics
•
•
http://docs.scipy.org/doc/scipy/reference/integrate.html
Interpolation
•
•
http://docs.scipy.org/doc/scipy/reference/optimize.html
http://docs.scipy.org/doc/scipy/reference/stats.html
Image processing
•
http://docs.scipy.org/doc/scipy/reference/ndimage.html
• • •
Python wrappers of GNU Scientific Library functions
•
Incomplete documentation for Python functions, but almost all of GSL is wrapped, so refer to GSL documentation.
•
Most functionality implemented in SciPy
PyGSL: http://pygsl.sourceforge.net/ GSL: http://www.gnu.org/software/gsl/
• • •
Try SciPy first, if you can’t find what you need try PyGSL More comprehensive and sometimes more tested, but less ‘Pythonic’ e.g. Monte Carlo integration
• •
http://rpy.sourceforge.net/
• •
http://www.sagemath.org/
Wraps R – a statistics analysis language
•
many advanced stats capabilities but quite specialised
Python-based mathematics software
•
replacement for Maple, Mathematica
•
Planetary nebulae positions and velocities in edge-on galaxy NGC 891
•
Need to:
• • • •
read in data make plot of positions, colour coded by velocity plot velocity histogram fit projected rotation curve to full dataset
An introduction to scientific programming with
Part 3: Python for observers
• •
http://web.mac.com/npirzkal/Scisoft Quick way to install many of the most used data reduction packages:
•
IRAF, PyRAF, MIDAS, Python and Python extensions and many more…
•
http://www.stsci.edu/resources/software_hardware/pyraf/stsci_python
• • • • •
Astronomy software provided by Space Telescope Science Institute
•
STScI also provide STSDAS and TABLES packages for IRAF
PyFITS PyRAF MultiDrizzle Numdisplay
•
http://www.stsci.edu/resources/software_hardware/pyfits
•
Read, write and manipulate all aspects of FITS files
• • • •
• •
extensions headers images tables
Low-level interface for details High-level functions for quick and easy use
>>> >>> >>> >>>
import pyfits imgname = ‘data/2MASS_NGC_0891_K.fits’ img = pyfits.getdata(imgname) img
array([[
0. , -999.00860596, [-999.00860596, -999.00860596, [-999.00860596, -999.00860596, ..., [-999.00860596, -999.00860596, [-999.00860596, -999.00860596, [-999.00860596, -999.00860596,
0. , 0. , ..., -999.00860596, -999.00860596], -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596], -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596], -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596], -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596], -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596]], dtype=float32)
>>> img.mean() -8.6610549999999993 >>> img[img > -99].mean() 0.83546290095423026 >>> numpy.median(img) 0.078269213438034058
>>> x = 348; y = 97 >>> delta = 5 >>> print img[y-delta:y+delta+1, ... x-delta:x+delta+1].astype(numpy.int) [[ 1 [ 2 [ 1 [ 1 [ 2 [ 3 [ 2 [ 1 [ 0 [ 0 [ -1
• • •
1 2 4 6 7 7 4 1 0 0 0
1 4 11 23 33 27 12 2 0 0 1
1 1 0 6 7 7 24 40 40 62 110 107 91 158 148 74 123 115 32 54 51 7 12 12 1 2 2 1 0 0 0 0 0
0 4 21 50 68 53 24 5 1 0 0
0 3 7 13 15 12 5 0 0 0 0
1 1 2 2 3 2 1 0 0 0 0
0 0 0 0 0 0 0 0 1 0 0
-2] -1] 0] 0] 0] 0] 0] 0] 0] 0] 0]]
y
row = y = first index column = x = second index numbering runs as normal (e.g. in ds9) BUT zero indexed!
x
>>> tblname = ‘data/N891PNdata.fits’ >>> d = pyfits.getdata(tblname) >>> d.names ('x0', 'y0', 'rah', 'ram', 'ras', 'decd', 'decm', 'decs', 'wvl', 'vel', 'vhel', 'dvel', 'dvel2', 'xL', 'yL', 'xR', 'yR', 'ID', 'radeg', 'decdeg', 'x', 'y')
>>> d.x0 array([ 928.7199707 , 532.61999512, 968.14001465, 519.38000488,… 1838.18994141, 1888.26000977, 1516.2199707 ], dtype=float32)
>>> d.field(‘x0’)
# case-insensitive
array([ 928.7199707 , 532.61999512, 968.14001465, 519.38000488,… 1838.18994141, 1888.26000977, 1516.2199707 ], dtype=float32)
>>> select = d.x0 < 200 >>> dsel = d[select] >>> print dsel.x0 [ 183.05000305 165.55000305 192.58000183 157.02999878 175.19000244]
# can select rows all together 138.47999573 160.1499939
158.02999878 161.1000061
140.96000671 136.58999634
>>> h = pyfits.getheader(imgname) >>> print h SIMPLE = BITPIX = NAXIS = NAXIS1 = NAXIS2 = BLOCKED = EXTEND = BSCALE = BZERO = ORIGIN = CTYPE1 = CTYPE2 = CRPIX1 = CRPIX2 = CRVAL1 = CRVAL2 = CDELT1 = CDELT2 = CROTA2 = EQUINOX = KMAGZP = COMMENTC= SIGMA = COMMENT1=
'2MASS ' 'RA---SIN' 'DEC--SIN'
T -32 2 1000 1200 T / TAPE MAY BE BLOCKED IN MULTIPLES OF 2880 T / TAPE MAY HAVE STANDARD FITS EXTENSIONS 1. 0. / 2MASS Survey Camera
500.5 600.5 35.63922882 42.34915161 -0.0002777777845 0.0002777777845 0. 2000. 20.07760048 / V3 Photometric zero point calibration 'CAL updated by T.H. Jarrett, IPAC/Caltech' 1.059334397 / Background Residual RMS noise (dn) '2MASS mosaic image'
COMMENT2= 'created by T.H. Jarrett, IPAC/Caltech’
>>> h[‘KMAGZP’] 20.077600480000001 # Use h.items() to iterate through all header entries
>>> newimg = sqrt((sky+img)/gain + rd_noise**2) * gain >>> newimg[(sky+img) < 0.0] = 1e10 >>> hdr = h.copy() # copy header from original image >>> hdr.add_comment(‘Calculated noise image’) >>> filename = ‘sigma.fits’ >>> pyfits.writeto(filename, newimg, hdr)
# create new file
>>> pyfits.append(imgname, newimg, hdr) # add a new FITS extension >>> pyfits.update(filename, newimg, hdr, ext) # update a file # specifying a header is optional, # if omitted automatically adds minimum header
>>> import pyfits >>> import numpy as np >>> # create data >>> a1 = numpy.array(['NGC1001', 'NGC1002', 'NGC1003']) >>> a2 = numpy.array([11.1, 12.3, 15.2]) >>> # make list of pyfits Columns >>> cols = [] >>> cols.append(pyfits.Column(name='target', format='20A', array=a1)) >>> cols.append(pyfits.Column(name='V_mag', format='E', array=a2)) >>> # create HDU and write to file >>> tbhdu=pyfits.new_table(cols) >>> tbhdu.writeto(â&#x20AC;&#x2122;table.fitsâ&#x20AC;&#x2122;) # these examples are for a simple FITS file containing just one # table or image but with a couple more steps can create a file # with any combination of extensions (see the PyFITS manual online)
>>> f = pyfits.open(tblname) >>> f.info() Filename: data/N891PNdata.fits No. Name Type Cards Dimensions 0 PRIMARY PrimaryHDU 4 () 1 BinTableHDU 52 223R x 22C E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, >>> table = f[1]
Format uint8 [E, E, E, E, E, E]
# data extension number 1 (can also use names)
>>> d = f[1].data # data, same as returned by pyfits.getdata() >>> h = f[1].header # header, same as returned by pyfits.getheader () >>> # make any changes >>> f.writeto(othertblname) >>> >>> >>> >>>
# writes (with changes) to a new file
f = pyfits.open(tblname, mode=â&#x20AC;&#x2DC;updateâ&#x20AC;&#x2122;) # to change same file # make any changes f.flush() # writes changes back to file f.close() # writes changes and closes file
•
http://www.stsci.edu/resources/software_hardware/pyraf
•
Command line to replace cl, allows most normal IRAF commands and Python at same prompt
•
Can use IRAF tasks in Python scripts instead of having to create awkward cl scripts (or worse SPP)
• • • • •
Command and filename completion Edit line and access history easily (like ecl or bash) Use just as friendlier cl prompt or use Python whenever you want Transfer data between IRAF tasks and Python Use brackets for tasks when you want it to behave like Python
--> imstat 2MASS_NGC_0891_K.fits # IMAGE NPIX 2MASS_NGC_0891_K.fits 1200000
MEAN -8.661
--> fname = "data/2MASS_NGC_0891_K.fits" --> imstat fname # IMAGE NPIX MEAN Error reading image fname ...
STDDEV 99.44
STDDEV
MIN -1001.
MIN
MAX 7207.
MAX
--> imstat(fname) # IMAGE NPIX MEAN STDDEV MIN MAX data/2MASS_NGC_0891_K.fits 1200000 -8.661 99.44 -1001. 7207.
•
Many IRAF tasks create output images or tables on disk, or just print to screen, so don’t need to pass information back (see later for this)
stsdas # note can’t unload packages improject(sky_file_2D, sky_file_1D, projaxis=2, average='no') imcalc(sky_file_1D, sky_file_1D, 'im1*%f'%apwidthratio) # calculate effective gain and ron due to combined images gain = 1.91; ron = 5.41 gain_eff = gain * ncombine ron_eff = ron * sqrt(ncombine) # sig = sci + sky imcalc('%s,%s'%(sci_file, sky_file_1D), sig_file, 'im1+im2') # sig = sqrt(sig * gain + ron**2) / gain equation = 'sqrt(max(im1,0.0)/%(g)8.5f + %(r2)8.5f/%(g2)8.5f)' equation = equation%{'g': gain_eff, 'r2': ron_eff**2, 'g2': gain_eff**2} imcalc(sig_file, sig_file, equation)
• •
Use IRAF tasks in Python scripts Just import iraf object from pyraf module
from pyraf import iraf from glob import glob images = glob(‘*sci.fits’) for img in images: iraf.imstat(img) newimg = img.replace(‘sci’, ‘sig’) iraf.imcalc(img, newimg, ’sqrt(im1)')
•
Can specify ‘default’ task parameters in neat fashion, instead of having to include on every call of a task
iraf.imstat.nclip = 3 iraf.imstat.lsigma = 5 iraf.imstat.usigma = 5 # now every time I use imstat it uses sigma clipping iraf.imstat(im1) iraf.imstat(im2) iraf.imstat(im3) # can revert to task defaults by unlearning iraf.unlearn(‘imstat’) # note task name is a string
•
Useful to make shortcuts
# shortcut for a long task name crrej = iraf.stsdas.hst_calib.wfpc.crrej crrej.mask = ‘mymask.fits’ crrej.sigma = 5 crrej(in1, out1) crrej(in2, out2) crrej(in3, out3)
• • •
IRAF outputs lots of useful data to screen - how to get at it? Some tasks need user input - would be nice to automate these PyRAF defines extra parameters Stdout and Stdin for all tasks
• •
Stdout can be a filename, file object, or 1 (one) to direct to a list Stdin can be a filename, file object, or string
for img in images: # get a list with a string for each line of output output = iraf.imstat(img, format=‘no’,fields=‘midpt’,Stdout=1) # output can then be parsed as necessary to get at info med = float(output[0]) newimg = img.replace(‘sci’, ‘sub’) iraf.imcalc(img, newimg, ’im1 - %f’%med)
An introduction to scientific programming with
The End