Programming Real-Time Sound in Python
Abstract
:1. Introduction
1.1. Related Work
1.2. On Real-Time Processing
1.3. Structure of the Paper
2. Interpreted Approach
3. Code Speedup
3.1. Numba
3.2. Cython
- c_low_pass.c contains the procedure within the C function lp.
- low_pass.pyx binds the C file to the corresponding Python functions. From it, Cython compiles the corresponding module low_pass.
- test_lp.py calls the low_pass module functions.
3.3. Comparisons & Benchmarking
- In general, code refactoring is easier and faster in Numba than Cython.
- Numba performs well if arithmetic operations are repeatedly computed without communicating with Python objects at runtime.
- Cython code is compiled once (AOT), while Numba optimizes a script at each new execution (JIT).
- Cython requires writing C instructions.
- Cython allows for debugging and profiling the code after translation.
- Cython allows embedding existing code written in C or other languages.
4. Applications to Virtual Analog
4.1. Ring Modulator
Algorithm 1: Ring modulator core processing loop. |
1: repeat 2: read ${v}_{m}$, ${v}_{c}$ 3: compute ${v}_{1}=\frac{{R}_{m}}{1+C{F}_{s}{R}_{m}}\left(\frac{{v}_{m}}{{R}_{m}}+{i}_{1}-\frac{g\left({v}_{4}\right)}{2}+\frac{g\left({v}_{5}\right)}{2}-\frac{g\left({v}_{6}\right)}{2}+\frac{g\left({v}_{7}\right)}{2}\right)+{v}_{1}$ 4: compute ${v}_{2}=\frac{{R}_{a}}{1+C{F}_{s}{R}_{a}}\left({i}_{2}+\frac{g\left({v}_{4}\right)}{2}-\frac{g\left({v}_{5}\right)}{2}-\frac{g\left({v}_{6}\right)}{2}+\frac{g\left({v}_{7}\right)}{2}\right)+{v}_{2}$ 5: compute ${v}_{3}=\frac{{R}_{i}}{1+{C}_{p}{F}_{s}{R}_{i}}\left(g\left({v}_{4}\right)+g\left({v}_{5}\right)-g\left({v}_{6}\right)-g\left({v}_{7}\right)\right)+{v}_{3}$ 6: compute ${v}_{4}=\frac{{v}_{1}}{2}-\frac{{v}_{2}}{2}-{v}_{3}-{v}_{c}$ 7: compute ${v}_{5}=-\frac{{v}_{1}}{2}+\frac{{v}_{2}}{2}-{v}_{3}-{v}_{c}$ 8: compute ${v}_{6}=\frac{{v}_{1}}{2}+\frac{{v}_{2}}{2}+{v}_{3}+{v}_{c}$ 9: compute ${v}_{7}=-\frac{{v}_{1}}{2}-\frac{{v}_{2}}{2}+{v}_{3}+{v}_{c}$ 10: compute ${i}_{1}=-\frac{1}{L{F}_{s}}{v}_{1}+{i}_{1}$ 11: compute ${i}_{2}=-\frac{1}{L{F}_{s}}{v}_{2}+{i}_{2}$ 12: until ever. |
4.2. Voltage-Controlled Filter
Algorithm 2: Voltage-controlled filter core processing loop. |
1: repeat 2: read ${v}_{IN}$ 3: repeat 4: compute ${v}_{OUT}^{*}={v}_{OUT}$ 5: compute ${u}_{1}=tanh\left(({v}_{OUT}^{*}-{v}_{IN})/2{V}_{T}\right)$ 6: compute ${v}_{C1}={I}_{0}({u}_{2}+{u}_{1})/\left(4C{F}_{S}\right)+{s}_{1}$ 7: compute ${u}_{2}=tanh\left(({v}_{C2}-{v}_{C1})/2{V}_{T}\right)$ 8: compute ${v}_{C2}={I}_{0}({u}_{3}-{u}_{2})/\left(4C{F}_{S}\right)+{s}_{2}$ 9: compute ${u}_{3}=tanh\left(({v}_{C3}-{v}_{C2})/2{V}_{T}\right)$ 10: compute ${v}_{C3}={I}_{0}({u}_{4}-{u}_{3})/\left(4C{F}_{S}\right)+{s}_{3}$ 11: compute ${u}_{4}=tanh\left(({v}_{C4}-{v}_{C3})/2{V}_{T}\right)$ 12: compute ${v}_{C4}={I}_{0}({u}_{5}-{u}_{4})/\left(4C{F}_{S}\right)+{s}_{4}$ 13: compute ${u}_{5}=tanh({v}_{C4}/6\gamma )$ 14: compute ${v}_{OUT}=(K+1/2){v}_{C4}$ 15: until$|{v}_{OUT}-{v}_{OUT}^{*}|<{10}^{-4}\left|{v}_{OUT}^{*}\right|$ 16: compute ${s}_{1}={v}_{C1}/\left(2{F}_{S}\right)+{s}_{1}$ 17: compute ${s}_{2}={v}_{C2}/\left(2{F}_{S}\right)+{s}_{2}$ 18: compute ${s}_{3}={v}_{C3}/\left(2{F}_{S}\right)+{s}_{3}$ 19: compute ${s}_{4}={v}_{C4}/\left(2{F}_{S}\right)+{s}_{4}$ 20: until ever. |
5. Discussion
6. Conclusions
- those parts of the code which deal with environment setup, type conversions, user interaction, signal analysis and interface development can be left to the interpreter;
- procedures that do not need to intensively interact with the program workspace can be compiled with Numba;
- procedures, which, conversely, must often interact with the program workspace in terms of e.g., memory allocation or signal resampling and filtering, can be compiled with Cython; and,
- finally, already existing procedures written in C or other languages can also be efficiently cythonized.
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Appendix A. Code Listings
def callback (i_d , frame_count , t_info , f): data = wavefile . readframes (frame_count) samples = pcm2float (data) y = process (data) out = float2pcm (y) return (out, pyaudio. paContinue) |
def process ( data ): global last_sample b = 1- alpha y= np. arange ( CHUNK_SIZE , dtype = float ) y[0] = alpha * data [0] + b* last_sample for i in range (1, CHUNK_SIZE -1): y[i] = alpha * data [i] + b*y[i -1] last_sample = y[ CHUNK_SIZE -1] return y |
Function : try_global Hit time unit : 3.41*10^ -7 s Total time : 0.1037 s Line # Hits Time Per Hit % Time Line Contents ========================================================================== 1 1 3.0 3.0 0.0 a = 0 1 def try_global (): 2 global a 3 1 3.0 3.0 0.0 i = 0 4 100 ,001 85 ,289.0 0.9 28.0 while i < 100000: 5 100 ,000 92 ,719.0 0.9 30.5 i += 1 6 100 ,000 126 ,083.0 1.3 41.5 a = math .sin(i) |
from numba import jit , float32 import math a=0. @jit ( float32 ( float32 ), nopython = True ) def numba_global (a): i=0 while i < 100000: i+=1 a = math . sin(i) return a # Execution and benchmark of the refactored function Function : wrap_numba_global Hit time unit : 3.41*10^ -7 s Total time : 0.0474 s Line # Hits Time Per Hit % Time Line Contents ============================================================ 1 def wrap_w_numba_global (): 2 global a 3 1 129 ,328.0 129 ,328.0 93.0 wrap_numba_global () 4 1 3.0 3.0 0.0 a = 1. 5 1 2183.0 2183.0 1.6 wrap_numba_global () |
Function : cython_try_global Hit time unit : 3.41*10^ -7 s Total time : 0.0526 s Line # Hits Time Per Hit % Time Line Contents ============================================================ 1 1 3.0 3.0 0.0 cdef float a = 0 2 def cython_try_global (): 3 global a 4 1 4.0 4.0 0.0 cdef int i = 0 4 1 1.0 1.0 0.0 while i < 100000: 8 100 ,000 55 ,087.0 0.6 37.4 i += 1 9 100 ,000 92 ,154.0 0.9 62.6 a = math .sin(i) |
=========================== c_low_pass .c =========================== double last_sample = 0; double b = 0; void lp( double * data , int samples , double * a){ b = 1 - a; data [0] = a* data [0] + b* last_sample ; for ( int i = 1; i < samples ; i++) { data [i] = a* data [i] + b* data [i -1]; } last_sample = data [ samples -1]; return ; } =========================== low_pass .pyx =========================== import cython ... # declare interface to C code cdef extern void lp( double * data , int samples , double * a) # c- array that contains samples cdef double * cdata # function compiled as module def process (np. ndarray [ double ] data , int samples , double alfa ): ... lp(cdata , samples , a) ... return data =========================== test_lp .py =========================== import low_pass ... # ’’a’’ is the user parameter def process ( data ): return low_pass . process (data , CHUNK_SIZE , a) |
# numba decorator with static type declaration and mode @jit ( types . Tuple (( returned_values_types ))( parameters_types ), nopython = True ) def compute_numba_low (c_data , m_data , u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2 ): for i in range ( FRAMES_PER_BUFFER * over_f ): equation1 ... equation9 data [i] = equation2_result return data ,u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2 # numba wrapper def compute_numba (c_data , m_data ): global u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2 ,zi resample of two buffers to oversampling rate using 0- padding data ,u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2 = compute_numba_low (c_data ,m_data , u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2) LPF on returned data return data [:: over_f ] # take one sample each over_f samples |
=========================== diode_cython . pyx =========================== # this file needs to be compiled # static type declaration of global variables cdef double u1 = 0. ... # parameter type def . e.g.,( np. ndarray [ double , ndim =1, mode ="c"] c_data not None ,...) def compute_c ( c_data , m_data , FRAMES_PER_BUFFER , T, ovr_f , a, b, zi ): global u1 ,u2 ,u3 ,u4 ,u5 ,u6 ,u7 ,i1 ,i2 resample of two buffers to oversampling rate using 0- padding samples = FRAMES_PER_BUFFER * ovr_f for i in range (0, samples ): equation1 ... equation9 c_data [i] = equation2_result LPF on returned data return c_data [:: ovr_f ] # take one sample each ovr_f samples |
=========================== diode .pyx =========================== # this file needs to be compiled # declare interface to external C code cdef extern void ring ( double * c, double * m, int s, float T) declaration of C arrays and memory space allocation # parameter type def. e.g.,( np. ndarray [ double ] c_data ,...) def process (c_data , m_data , FRAMES_PER_BUFFER , T, ovr_f , a, b, zi ): cdef unsigned int i resample of two buffers to oversampling rate using 0- padding move data from python np. arrray to C memory located # call the C function compute_c (cdata ,mdata , FRAMES_PER_BUFFER *ovr_f , T) read modified data into np. array LPF on returned data return c_data [:: ovr_f ] # take one sample each ovr_f samples |
=========================== c_diode .c =========================== // declare global variables double u1 = 0; ... void ring ( double * carr_in , double * mod_in , int samples , float T){ int i = 0; for (i = 0; i < samples ; i++) { equation1 ... equation9 carr_in [i] = equation2_result } return ; } |
=========================== test_diode .py =========================== import diode ... ovr_f = 30 # oversampling factor T = 1./( s_rate * ovr_f ) b,a = butt_low ( sample_rate /ovr_f , s_rate , 5) zi = signal . lfilter_zi (b,a) def process (c_in , m_in ): global zi return diode . process (c_in , m_in , CHUNK_SIZE ,T, ovr_f , a, b, zi) |
References
- McCartney, J. Rethinking the Computer Music Language: SuperCollider. Comput. Music J. 2002, 26, 61–68. [Google Scholar] [CrossRef]
- Adams, A.T.; Latulipe, C. Survey of Audio Programming Tools. In CHI ‘13 Extended Abstracts on Human Factors in Computing Systems; CHI EA ’13; ACM: New York, NY, USA, 2013; pp. 781–786. [Google Scholar] [CrossRef]
- Morreale, F.; McPherson, A.P. Design for longevity: Ongoing use of instruments from NIME 2010–14. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME’17), Copenhagen, Denmark, 15–19 May 2017; pp. 192–197. [Google Scholar]
- Krishan Kumar, S.D. Programming Languages: A Survey. Int. J. Recent Innov. Trends Comput. Commun. 2017, 5, 307–313. [Google Scholar]
- Guo, P. Python is Now the Most Popular Introductory Teaching Language at Top U.S. Universities. 2014. Available online: https://cacm.acm.org/blogs/blog-cacm/176450 (accessed on 18 June 2020).
- Mason, M.; Cooper, G.; Raadt, M. Trends in Introductory Programming Courses in Australian Universities Languages, Environments and Pedagogy. In Proceedings of the Australasian Computing Education Conference (ACE2012), Melbourne, Australia, 31 January–3 February 2012; CRPIT. Raadt, M., Carbone, A., Eds.; ACS: Melbourne, Australia; Volume 123, pp. 33–42. [Google Scholar]
- Downey, A.B. Think DSP: Digital Signal Processing in Python, 1st ed.; O’Reilly Media, Inc.: Newton, MA, USA, 2016. [Google Scholar]
- Sepúlveda, S.; Reyes, P.; Weinstein, A. Visualizing physiological signals in real-time. In Proceedings of the 14th Python in Science Conference (SciPy 2015), Austin, TX, USA, 6–12 July 2015; pp. 190–194. [Google Scholar] [CrossRef]
- Behnel, S.; Bradshaw, R.; Citro, C.; Dalcin, L.; Seljebotn, D.S.; Smith, K. Cython: The Best of Both Worlds. Comput. Sci. Eng. 2011, 13, 31–39. [Google Scholar] [CrossRef]
- Wright, A.; Damskägg, E.P.; Juvela, L.; Välimäki, V. Real-Time Guitar Amplifier Emulation with Deep Learning. Appl. Sci. 2020, 10, 766. [Google Scholar] [CrossRef][Green Version]
- Martínez Ramírez, M.A.; Benetos, E.; Reiss, J.D. Deep Learning for Black-Box Modeling of Audio Effects. Appl. Sci. 2020, 10, 638. [Google Scholar] [CrossRef][Green Version]
- De Pra, Y.; Fontana, F.; Simonato, M. Development of real-time audio applications using Python. In Proceedings of the XXII Colloquium of Musical Informatics, Udine, Italy, 22–23 February 2018; pp. 226–231. [Google Scholar]
- Mathews, M.V.; Miller, J.E.; Moore, F.R.; Pierce, J.R.; Risset, J.C. The Technology of Computer Music; MIT Press: Cambridge, MA, USA, 1969; Volume 5. [Google Scholar]
- Vercoe, B.; Ellis, D. Real-time CSound: Software Synthesis with Sensing and Control. In Proceedings of the International Computer Music Conference, Glasgow, Scotland, 10–15 September 1990; pp. 209–211. [Google Scholar]
- Puckette, M. Pure Data: Another integrated computer music environment. In Proceedings of the International Computer Music Conference, Hong Kong, China, 19–24 August 1996; pp. 37–41. [Google Scholar]
- Cycling ’74. Max/MSP Website. 2020. Available online: https://cycling74.com/products/max/ (accessed on 18 June 2020).
- Wang, G.; Cook, P.R. ChucK: A Concurrent, On-the-Fly, Audio Programming Language. In Proceedings of the International Computer Music Conference, Singapore, 29 September–4 October 2003; pp. 219–226. [Google Scholar]
- Fober, D.; Orlarey, Y.; Letz, S. Faust Architectures Design and Osc Support. In Proceedings of the International Conference on Digital Audio Effects (DAFx-11), Paris, France, 19–23 September 2011; pp. 231–236. [Google Scholar]
- Glover, J.C.; Lazzarini, V.; Timoney, J. Python for Audio Signal Processing. In Proceedings of the Linux Audio Conference 2011, Maynooth, Ireland, 6–8 May 2011; pp. 107–114. [Google Scholar]
- McFee, B.; Raffel, C.; Liang, D.; Ellis, D.; McVicar, M.; Battenberg, E.; Nieto, O. librosa: Audio and Music Signal Analysis in Python. In Proceedings of the 14th Python in Science Conference, Austin, TX, USA, 6–12 July 2015; pp. 18–25. [Google Scholar] [CrossRef]
- Bellini, D. Expressive Digital Signal Processing (DSP) Package for Python. 2016. Available online: https://github.com/danilobellini/audiolazy (accessed on 18 June 2020).
- Wickert, M. Real-Time Digital Signal Processing using pyaudio_helper and the ipywidgets. In Proceedings of the 17th Python in Science Conference, Austin, TX, USA, 9–15 July 2018; pp. 91–98. [Google Scholar] [CrossRef]
- Belanger, O. Pyo, the Python DSP Toolbox. In Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, The Netherlands, 15–19 October 2016; ACM: New York, NY, USA, 2016; pp. 1214–1217. [Google Scholar] [CrossRef]
- ELK Audio. Audio Latency Demystified. 2020. Available online: https://elk.audio/audio-latency-demystified-part-ii/ (accessed on 18 June 2020).
- Bencina, R. Real-Time Audio Programming. 2011. Available online: http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing (accessed on 18 June 2020).
- Pham, H. PyAudio Website. 2006. Available online: https://people.csail.mit.edu/hubert/pyaudio (accessed on 18 June 2020).
- Stöter, F. Repository of Python Scientific Audio Packages. 2020. Available online: https://github.com/faroit/awesome-python-scientific-audio (accessed on 18 June 2020).
- Mitra, S.K. Digital Signal Processing. A Computer-Based Approach; McGraw-Hill: New York, NY, USA, 1998. [Google Scholar]
- Fontana, F.; Bozzo, E. Explicit Fixed-Point Computation of Nonlinear Delay-Free Loop Filter Networks. IEEE/ACM Trans. Audio Speech Lang. Process. 2018, 26, 1884–1896. [Google Scholar] [CrossRef]
- Kern, R. Line Profiler Code Repository. 2020. Available online: https://github.com/rkern/line_profiler (accessed on 18 June 2020).
- Lam, S.K.; Pitrou, A.; Seibert, S. Numba: A LLVM-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, Austin, TX, USA, 11 November 2015; pp. 1–6. [Google Scholar]
- The Computer Language Benchmarks Game. Benchmark Python vs. C++. 2020. Available online: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gpp-python3.html (accessed on 18 June 2020).
- Milde, D. Benchmark Numba vs. Cython. 2020. Available online: https://pybenchmarks.org/u64q/numba.php (accessed on 18 June 2020).
- Gorelick, M.; Ozsvald, I. High Performance Python: Practical Performant Programming for Humans; O’Reilly Media, Inc.: Newton, MA, USA, 2014. [Google Scholar]
- Ringle, E. Portaudio. 2020. Available online: https://github.com/EddieRingle/portaudio/blob/master/examples/ (accessed on 18 June 2020).
- Zölzer, U. DAFX: Digital Audio Effects; John Wiley & Sons: Hoboken, NJ, USA, 2011. [Google Scholar]
- Pedregosa, F. Memory Profiler Code Repository. 2020. Available online: https://github.com/pythonprofilers/memory_profiler (accessed on 18 June 2020).
- Fontana, F.; Civolani, M. Modeling of the EMS VCS3 Voltage-Controlled Filter as a Nonlinear Filter Network. IEEE Trans. Audio Speech Lang. Process. 2010, 18, 760–772. [Google Scholar] [CrossRef]
Processing Time | Memory Occupation | |
---|---|---|
Interpreted Python | 630.2 $\mathsf{\mu}$s/sample | 118 KB |
Cython annotations | 1.98 $\mathsf{\mu}$s/sample | 230 KB |
Cython + native C | 1.98 $\mathsf{\mu}$s/sample | 301 KB |
Numba | 1.93 $\mathsf{\mu}$s/sample | 219 KB |
Processing Time | Memory Occupation | |
---|---|---|
Interpreted Python | 199.3 $\mathsf{\mu}$s/sample | <1 KB |
Cython annotations | 6.5 $\mathsf{\mu}$s/sample | 16 KB |
Cython + native C | 0.14 $\mathsf{\mu}$s/sample | 12 KB |
Numba | 1.36 $\mathsf{\mu}$s/sample | 12 KB |
Load | Audio Only | GUI Operations | File Copy | ||||||
---|---|---|---|---|---|---|---|---|---|
Idle | Process | Total | Idle | Process | Total | Idle | Process | Total | |
Min (ms) | 0.005 | 0.039 | 0.044 | 0.005 | 0.037 | 0.042 | 0.005 | 0.038 | 0.044 |
Max (ms) | 36.5 | 8.7 | 36.8 | 33.9 | 31.6 | 40.9 | 27.9 | 29.9 | 29.9 |
Average (ms) | 1.31 | 0.14 | 1.44 | 1.32 | 0.13 | 1.44 | 1.35 | 0.09 | 1.45 |
Violations (%) | 11% | 0.17% | 11.2% | 11% | 0.8% | 11.8% | 11% | 0.15% | 11.2% |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
De Pra, Y.; Fontana, F. Programming Real-Time Sound in Python. Appl. Sci. 2020, 10, 4214. https://doi.org/10.3390/app10124214
De Pra Y, Fontana F. Programming Real-Time Sound in Python. Applied Sciences. 2020; 10(12):4214. https://doi.org/10.3390/app10124214
Chicago/Turabian StyleDe Pra, Yuri, and Federico Fontana. 2020. "Programming Real-Time Sound in Python" Applied Sciences 10, no. 12: 4214. https://doi.org/10.3390/app10124214