diff --git a/src/content.tex b/src/content.tex new file mode 100644 index 0000000..28a7ac8 --- /dev/null +++ b/src/content.tex @@ -0,0 +1,435 @@ + +\begin{abstract} + This paper proposes \lambdammm , a call-by-value, simply-typed lambda calculus-based intermediate representation for a music programming language, which handles synchronous signal processing, Additionally, it introduces a virtual machine and instruction set to execute \lambdammm . Digital signal processing is represented through a syntax that incorporates the internal states of delay and feedback into the lambda calculus. \lambdammm\ extends the lambda calculus, allowing users to construct generative signal processing graphs and execute them with consistent semantics. However, a challenge arises when handling higher-order functions, as users must determine whether execution occurs within the global environment or during DSP execution. This issue can potentially be resolved through multi-stage computation. + \end{abstract} + + \section{Introduction} + \label{sec:intro} + + + Many programming languages for sound and music have been developed, but only a few possess strongly formalized semantics. One language that is both rigorously formalized and practical is Faust \cite{Orlarey2004}, which combines blocks with inputs and outputs with five primitive operations: parallel, sequential, split, merge, and recursive connection. By providing basic arithmetic, conditionals, and delays as primitive blocks, almost any type of signal processing can be written in Faust. In a later extension, a macro based on a term rewriting system was introduced, allowing users to parameterize blocks with an arbitrary number of inputs and outputs \cite{graf2010}. + + This strong abstraction capability through formalization enables Faust to be translated to various backends, such as C, C++, Rust, and LLVM IR. On the other hand, Faust's Block Diagram Algebra (BDA) lacks theoretical and practical compatibility with common programming languages. Although it is possible to call external C functions in Faust, those functions are assumed to be pure functions that do not require heap memory allocation or deallocation. Therefore, while it is easy to embed Faust in another language, it is not easy to call another language from Faust. + + In addition, a macro for Faust is an independent term rewriting system that generates BDA based on pattern matching. As a result, the arguments for pattern matching are implicitly required to be integers, which can sometimes lead to compile-time errors, despite the fact that BDA does not distinguish between real and integer types. These implicit typing rules are not intuitive for novice users. + + Proposing a computational model for signal processing based on more generic computational models, such as lambda calculus, has the potential to enable interoperability between many different general-purpose languages, and also facilitate the appropriation of existing optimization methods and the implementation of compilers and run-time. + + Currently, it has been demonstrated that BDA can be converted to a general-purpose functional language using an arrow, a higher-level abstraction of monads \cite{gaster2018}. However, higher-order functions in general-purpose functional languages are often implemented based on dynamic memory allocation and deallocation, making them difficult to use in host languages designed for real-time signal processing. + + Additionally, Kronos \cite{norilo2015} and W-calculus \cite{arias2021} are examples of lambda calculus-based abstractions influenced by Faust. Kronos is based on the theoretical foundation of System-$F\omega$, a variation of lambda calculus in which types themselves can be abstracted (i.e., a function that takes a type as input and returns a new type can be defined). In Kronos, type calculations correspond to signal graph generation, while value calculations correspond to the actual processing. Delay is the only special primitive operation in Kronos, and feedback routing can be represented as a recursive function application in type calculations. + + W-calculus includes feedback as a primitive operation, along with the ability to access the value of a variable from the past (i.e., delay). W-calculus restricts systems to those that can represent linear-time-invariant processes, such as filters and reverberators, and it defines more formal semantics, aiming for automatic proofs of linearity and the identity of graph topologies. + + Previously, the author designed the music programming language \textit{mimium} \cite{matsuura2021a}. By incorporating basic operations such as delay and feedback into lambda calculus, signal processing can be concisely expressed while maintaining a syntax similar to that of general-purpose programming languages. Notably, \textit{mimium}'s syntax is designed to resemble the Rust programming language. + + One of the earlier issues with \textit{mimium} was its inability to compile code that contained combinations of recursive or higher-order functions with stateful functions involving delay or feedback, as the compiler could not determine the data size of the internal state used in signal processing. + + In this paper, I propose the syntax and semantics of \lambdammm, an extended call-by-value simply-typed lambda calculus, as a computational model intended to serve as an intermediate representation for \textit{mimium}. Additionally, I propose a virtual machine and its instruction set, based on Lua's VM, to execute this computational model in practice. Finally, I discuss both the challenges and potential of the current \lambdammm\ model: one challenge is that users must differentiate whether a calculation occurs in a global context or during actual signal processing; another is that run-time interoperability with other programming languages could be easier than in existing DSP languages. + + + \section{Syntax} + \label{sec:syntax} + + \input{syntax.tex} + + The types and terms of \lambdammm\ are defined in Figure \ref{fig:syntax_v}. + + Two terms are introduced in addition to the standard simply-typed lambda calculus: $delay\ n\ e_1\ e_2$, which refers to the previous value of $e_1$ by $e_2$ samples (with a maximum delay of $n$ to limit memory usage to a finite size), and $feed\ x.e$, an abstraction that allows the user to refer to the result of evaluating $e$ from one time unit earlier as $x$ during the evaluation of $e$ itself. + + \subsection{Syntactic Sugar of the Feedback Expression in \textit{mimium}} + \label{sec:mimium} + + The programming language \textit{mimium}, developed by the author, includes a keyword $self$ that can be used in function definitions to refer to the previous return value of the function. An example of a simple one-pole filter function, which mixes the input and the last output signal such that the sum of the input and feedback gains is 1, is shown in Listing \ref{lst:onepole}. This code can be expressed in \lambdammm\ as shown in Figure \ref{fig:onepole}. + + \begin{lstlisting}[float,floatplacement=H,label=lst:onepole,language=Rust,caption=\it Example of the code of one-pole filter in mimium.] + fn onepole(x,g){ + x*(1.0-g) + self*g + } + \end{lstlisting} + + \begin{figure}[ht] + \begin{equation*} + \centering + \begin{aligned} + let\ & onepole = \\ + & \ \lambda x. \lambda g.\ feed\ y.\ x *(1.0 - g) + y * g \ in\ ... + \end{aligned} + \end{equation*} + \caption{\label{fig:onepole}{\it Equivalent expression to Listing \ref{lst:onepole} in $\lambda_{mmm}$.}} + \end{figure} + + \subsection{Typing Rules} + \label{sec:typing} + + \input{typing.tex} + + Additional typing rules for the usual simply-typed lambda calculus are shown in Figure \ref{fig:typing}. + + The primitive types include a real number type, used in most signal processing, and a natural number type, which is used for the indices of delay. + + In W-calculus, which directly inspired the design of \lambdammm, function types can only take tuples of real numbers and return tuples of real numbers. This restriction prevents the definition of higher-order functions. While this limitation is reasonable for a signal processing language—since higher-order functions require data structures like closures that depend on dynamic memory allocation—it also reduces the generality of the lambda calculus. + + In \lambdammm, the problem of memory allocation for closures is delegated to the runtime implementation (see Section \ref{sec:vm}), allowing the use of higher-order functions. However, the $feed$ abstraction does not permit function types as either input or output. Allowing function types in the $feed$ abstraction would enable the definition of functions whose behavior could change over time. While this is theoretically interesting, there are no practical examples in real-world signal processing, and such a feature would likely complicate implementations further. + + \section{Semantics} + \label{sec:semantics} + + \input{semantics.tex} + + The excerpt of operational semantics of the \lambdammm\ is shown in Figure \ref{fig:semantics}. This big-step semantics is a conceptual explanation of the evaluation that, when the current time is $n$, the previous evaluation environment $t$ samples before can be referred to as $E^{n-t}$ , and that when the time < 0, the evaluation of any term is evaluated to the default value of its type (0 for the numeric types). + + Of course, if we tried to execute this semantics in a straightforward manner, we would have to redo the calculation from time 0 to the current time every sample, with saving all the variable environments at each sample. In practice, therefore, a virtual machine is defined that takes into account the internal memory space used by $delay$ and $feed$, and the \lambdammm\ terms are converted into instructions for that machine before execution. + + \section{VM Model and Instruction Set} + \label{sec:vm} + + A model for the virtual machine and its instruction set to run \\ \lambdammm\ is based on the VM for Lua version 5\cite{ierusalimschy2005}. + + When executing a computational model based on lambda calculus, the problem is how to handle a data structure called a closure that captures the variable environment where the inner function is defined, to refer the outer variables from the inner function context. If the dictionary data of names and values of variables are paired with inner function, implementation of the compiler (intepreter) is simple, but run-time performance is limited. + + On the contrary, a runtime performance can be improved by performing a process called closure transformation (or lambda lifting), which analyses all the names of outer variables referred by the inner function and transforms the inner function by adding argument so that the variables can be referred explicitly, but the compiler implementation of the transformation is relatively complex. + + The Lua VM takes an intermediate approach between these two by adding the VM instructions \texttt{GETUPVALUE} / \\ \texttt{SETUPVALUE}, which allows the outer variables to be referred dynamically at runtime. The implementation of compiler and VM using \textit{upvalue} is simpler than closure conversion, while at the same time preventing execution performance degradation, as outer variables can be referred via the call stack rather than on the heap memory unless the closure object escapes from the context of the original function\cite{nystrom2021}. + + Also, upvalue helps interoperations between other programming languages, as Lua can be easily embedded through C language API and when implementing external libraries in C, programmer can access to upvalues of Lua Runtime not only the stack values in C API. + + \subsection{Instruction Set} + \label{sec:instruction} + + VM Instructions for \lambdammm\ differs from the Lua VM in the following respects. + + \begin{enumerate} + + \item{Since mimium is a statically typed language unlike Lua, instructions for basic arithmetics are provided for each type.} + \item{The call operation is separated into the normal function call and the call of closure due to its static typing similarly, and also to handle higher-order statefull functions(See \ref{sec:vmstructure} for details). } + \item{If statements are realised by a combination of two instructions, \texttt{JMP} and \texttt{JMPIFNEG}, whereas the Lua VM uses a dedicated \texttt{TEST} instructions.} + \item{Instructions related to for loop, the \texttt{SELF} instruction used for object-oriented programming and the \texttt{TABLE}-related instructions for metadata references to variables are omitted in mimium as they are not used.} + \item{Instructions related to list-related data structures are also omitted in this paper, as the implementation of data structures such as tuples and arrays was omitted in the description of the \lambdammm\ in this paper.} + + \end{enumerate} + + Instructions in \lambdammm\ VM are 32bit data with operation tag and 3 operands. Currently, a bit width for the tag and each operands are all 8 bit\footnote[1]{Reason for this is that it is easy to implemented on \texttt{enum} data structure on Rust, a host language of the latest mimium compiler. Operands bitwidth and alignment may be changed in the future.}. + + The VM of \lambdammm\ is a register machine like the Lua VM (after version 5), although the VM has no real register but the register number simply means the offset index of the call stack from the base pointer at the point of execution of the VM. The first operand of most instructions is the register number in which to store the result of the operation. + + The list of instructions is shown in Figure \ref{fig:instruction} (basic arithmetic operations are partly omitted). The notation for the instruction follows the Lua VM paper \cite[p.13]{ierusalimschy2005}. From left to right, the name of operation, a list of operands, and pseudo-code of the operation. When using each of the three operands as unsigned 8 bits, they are denoted as \texttt{A B C}. When used with a signed integer, prefix \texttt{s} is added, and when the two operand fields are used as one 16 bits, an suffix \texttt{x} is added. For example, when B and C are merged and treated as signed 16 bits, they are denoted as \texttt{sBx}. + + In pseudo-code describing an functionality, \texttt{R(A)} means that data is moved in and out through the register (call stack) at the point of base pointer for current function + \texttt{A}. \texttt{K(A)} means that it retrieves the \texttt{A}-th number in the static variable field of the compiled program. \texttt{U(A)} means that referring \texttt{A}-th upvalue of the current function. + + In addition to Lua's Upvalue operation, 4 operations related to internal state variables over time, \texttt{GETSTATE}, \texttt{SETSTATE}, \\ \texttt{SHIFTSTATE} and \texttt{DELAY} are added to compile $delay$ and $feed$ expressions. + + \begin{figure*}[ht] + \tt + \small + \begin{tabular}{lll} + MOVE & A B & R(A) := R(B) \\ + MOVECONST & A B & R(A) := K(B) \\ + GETUPVALUE & A B & R(A) := U(B) \\ + \multicolumn{3}{l}{ + \textit{(SETUPVALUE does not exist)} + }\\ + GETSTATE* & A & R(A) := SPtr[SPos] \\ + SETSTATE* & A & SPtr[SPos] := R(A) \\ + SHIFTSTATE* & sAx & SPos += sAx \\ + DELAY* & A B C & R(A) := update\_ringbuffer(SPtr[SPos],R(B),R(C)) \\ + \multicolumn{3}{l}{ + \textit{ *((SPos,SPtr)= vm.closures[vm.statepos\_stack.top()].state } + }\\ + \multicolumn{3}{l}{ + \textit{\quad if vm.statepos\_stack is empty, use global state storage.)} + }\\ + JMP & sAx & PC +=sAx \\ + JMPIFNEG & A sBx & if (R(A)<0) then PC += sBx \\ + CALL & A B C & R(A),...,R(A+C-2) := program.functions[R(A)](R(A+1),...,R(A+B-1)) \\ + CALLCLS & A B C & vm.statepos\_stack.push(R(A)) \\ + \ & \ & R(A),...,R(A+C-2) := vm.closures[R(A)].fnproto(R(A+1),...,R(A+B-1)) \\ + \ & \ & vm.statepos\_stack.pop() \\ + CLOSURE & A Bx & vm.closures.push(closure(program.functions[R(Bx)])) \\ + & & R(A) := vm.closures.length - 1 \\ + CLOSE & A & close stack variables up to R(A)\\ + RETURN & A B & return R(A), R(A+1)...,R(A+B-2) \\ + ADDF & A B C & R(A) := R(B) as float + R(C) as float\\ + SUBF & A B C & R(A) := R(B) as float - R(C) as float\\ + MULF & A B C & R(A) := R(B) as float * R(C) as float\\ + DIVF & A B C & R(A) := R(B) as float / R(C) as float\\ + ADDI & A B C & R(A) := R(B) as int + R(C) as int \\ + \ & + \multicolumn{2}{l}{ + \textit{...Other basic arithmetics continues for each primitive types...} + } + \end{tabular} + \caption{\label{fig:instruction}{\it Instruction sets for VM to run $\lambda_{mmm}$.}} + \end{figure*} + + \subsection{Overview of the VM Structure} + \label{sec:vmstructure} + + The overview of a data structure of the virtual machine, the program and the instantiated closure for \lambdammm\ is shown in Figure \ref{fig:vmstructure}. In addition to the normal call stack, the VM has a storage area for managing internal state data for feedback and delay. + + This storage area is accompanied by data indicating the position from which the internal state is retrieved by the \texttt{GETSTATE} / \texttt{SETSTATE} instructions. This position is modified by \\ \texttt{SHIFTSTATE} operation back and forth. The actual data in the state storage memory are statically layed out at compile time by analyzing function calls that include references to \texttt{self}, call of \texttt{delay} and the functions which will call such statefull functions recursively. \texttt{DELAY} operation takes 2 inputs, B for an input and C for the delay time in samples. + + However, in the case of higher-order functions that receive a function as an argument and return another function, the layout of the internal state of the given function is unknown at the compilation, so an internal state storage area is created for each instantiated closure separately from the global storage area held by the VM instance itself. The VM have an another stack to keep the pointer to state storage. Each time \texttt{CALLCLS} used, VM pushes a pointer to the state storage of instantiated closure to the state stack and, at the end of the closure call, VM pops out the state pointer from the stack. + + Instantiated closures also hold the storage area of upvalues. Until the closure exits the context of parent function (such a closure is called ``Open Closure''), upvalues holds a negative offset on the stack at the ongoing execution. This offset value can be determined at compile time, the offset is stored in the function prototype in the program. Also, not only local variables, upvalue may refer to parent funtion's upvalue (this situation can happens when at least 3 functions are nested). So the array of upvalue indexes in the function prototype holds a pair of tag whether it is local stack value or further upvalue and its index (negative offset of stack or parent function's upvalue index). + + For instance, if the Upvalue indexes in the program were like \texttt{[upvalue(1),local(3)]}, \texttt{GETUPVALUE 6 1} means that, take \texttt{3} from the upvalue indexes 1 and get value from \texttt{R(-3)} over the base pointer and store it to \texttt{R(6)}. + + When the closure escapes from the original function with \\ \texttt{RETURN} instruction, inserted \texttt{CLOSE} instruction \\ the \texttt{RETURN} instruction moves actual upvalues from the stack into somewhere on the heap memory. This upvalues may be referred from multiple locations when using nested closures, and some form of garbage collection needed to free memory after they are no longer referred. + + In the current specification, the paradigm is call-by-value and reassignment expression does not exist, therefore, \texttt{SETUPVALUE} instruction does not exist in \lambdammm\ VM. This difference also make a difference to the implemention of open upvalue in the closure because the open upvalue should be shared memory cell which maybe recursively converted into memory cell of closed value when the \texttt{CLOSE} instruction is called. + + \begin{figure*}[ht] + \centerline{\includegraphics[width=\hsize]{lambdammm_vm_structure}} + \caption{\label{fig:vmstructure}{\it Overview of the virtual machine, program and instantiated closures for \lambdammm.}} + \end{figure*} + + \subsection{Compilation to the VM Instructions} + + \begin{lstlisting}[float,floatplacement=H,label=lst:bytecodes_onepole,caption=\it Compiled VM instructions of one-pole filter example in Listing \ref{lst:onepole}] + CONSTANTS:[1.0] + fn onepole(x,g) state_size:1 + MOVECONST 2 0 // load 1.0 + MOVE 3 1 // load g + SUBF 2 2 3 // 1.0 - g + MOVE 3 0 // load x + MULF 2 2 3 // x * (1.0-g) + GETSTATE 3 // load self + MOVE 4 1 // load g + MULF 3 3 4 // self * g + ADDF 2 2 3 // compute result + GETSTATE 3 // prepare return value + SETSTATE 2 // store to self + RETURN 3 1 + \end{lstlisting} + + Listing \ref{lst:bytecodes_onepole} shows an basic example when the mimium code in Listing \ref{lst:onepole} is compiled into VM bytecode. When \texttt{self} is referred, the value is obtained with the \texttt{GETSTATE} instruction, and the internal state is updated by storing the return value with the \\ \texttt{SETSTATE} instruction before returning the value with \texttt{RETURN} from the function. Here, the actual return value is obtained by the second \texttt{GETSTATE} instruction in order to return the initial value of the internal state when time=0. + + For example, when a time counter is written as \texttt{| | \{self + 1\}}, it is the compiler's design choice whether the return value of time=0 should be 0 or 1 though the latter does not strictly follow the semantics E-FEED in Figure \ref{fig:semantics}. If the design is to return 1 when time = 0, the second \texttt{GETSTATE} instruction can be removed and the value for the \texttt{RETURN} instruction should be \texttt{R(2)}. + + A more complex example code and its expected bytecode instructions are shown in Listing \ref{lst:fbdelay} and Listing \ref{lst:bytecodes_fbdelay}. The codes define delay with a feedback as \texttt{fbdelay}, the other function \texttt{twodelay} uses two feedback delay with different parameters, and \texttt{dsp} finally uses two \texttt{twodelay} function. + + Each after the referring to \texttt{self} through \texttt{GETSTATE} instruction, or call to the other statefull function, \\ \texttt{SHIFTSTATE} instruction inserted to move the position of state storage forward to prepare the next non-closure function call. Before exiting function, the state position is reset to the same position as that the current function context has begun by \texttt{SHIFTSTATE} (A sum of the operand for \texttt{SHIFTSTATE} in a function must be always 0). Figure \ref{fig:fbdelay_spos} shows how the state position moves by \texttt{SHIFT-}\\\texttt{STATE} operations during the execution of \texttt{twodelay} function. + + By describing an internal state as a relative position in the state storage, the state data can be expressed as a flat array, which makes the implementation of the compiler simple, not like a tree structure that need to analyze a call tree from the root to generate as in the previous implementation of mimium. This is similar to upvalue makes the implementation of the compiler simpler by describing free variables as relative positions on the call stack. + + \begin{lstlisting}[float,floatplacement=H,label=lst:fbdelay,language=Rust,caption=\it Example code that combines self and delay without closure call.] + fn fbdelay(x,fb,dtime){ + x + delay(1000,self,dtime)*fb + } + fn twodelay(x,dtime){ + fbdelay(x,dtime,0.7) + +fbdelay(x,dtime*2,0.8) + } + fn dsp(x){ + twodelay(x,400)+twodelay(x,800) + } + \end{lstlisting} + + \begin{lstlisting}[float,floatplacement=H,label=lst:bytecodes_fbdelay,caption=\it Compiled VM instructions of feedback delay example in Listing \ref{lst:fbdelay}] + CONSTANTS:[0.7,2,0.8,400,800,0,1] + fn fbdelay(x,fb,dtime) state_size:2 + MOVE 3 0 //load x + GETSTATE 4 //load self + SHIFTSTATE 1 //shift Spos + DELAY 4 4 2 //delay(_,_,_) + MOVE 5 1 // load fb + MULF 4 4 5 //delayed val *fb + ADDF 3 3 4 // x+ + SHIFTSTATE -1 //reset SPos + GETSTATE 4 //prepare result + SETSTATE 3 //store to self + RETURN 4 1 //return previous self + + fn twodelay(x,dtime) state_size:4 + MOVECONST 2 5 //load "fbdelay" prototype + MOVE 3 0 + MOVE 4 1 + MOVECONST 5 0 //load 0.7 + CALL 2 3 1 + SHIFTSTATE 2 //2=state_size of fbdelay + MOVECONST 3 5 //load "fbdelay" prototype + MOVE 4 0 + MOVECONST 5 1 //load 2 + MULF 4 4 5 + MOVECONST 5 0 //load 0.7 + CALL 3 3 1 + ADDF 3 3 4 + SHIFTSTATE -2 + RETURN 3 1 + + fn dsp (x) + MOVECONST 1 6 //load "twodelay" prototype + MOVE 2 0 + MOVECONST 3 3 //load 400 + CALL 1 2 1 + SHIFTSTATE 4 //4=state_size of twodelay + MOVECONST 2 6 + MOVE 2 3 //load "twodelay" prototype + MOVE 3 0 + MOVECONST 3 4 //load 400 + CALL 2 2 1 + ADD 1 1 2 + SHIFTSTATE -4 + RETURN 1 1 + \end{lstlisting} + + \begin{figure}[ht] + \centerline{\includegraphics[width=0.7\hsize]{fbdelay_spos}} + \caption{\label{fig:fbdelay_spos}{\it Image of how the state position moves while executing \texttt{twodelay} function in Listing \ref{lst:bytecodes_fbdelay}.}} + \end{figure} + + + Listing \ref{lst:filterbank_good} shows an example of a higher-order function \\ \texttt{filterbank} that takes another function \texttt{filter} that takes an input and a frequency as an argument, duplicates \texttt{n} of filter, and adds them together. Note that in the previous specification of mimium in \cite{matsuura2021a}, the binding of new variable and destructive assignment were the same syntax (\texttt{x = a}) but the syntax for the variable binding has changed to use \texttt{let} keyword. Also, because the semantics is call-by-value paradigm, reassignment syntax never be used in the current implementation. + + The previous mimium compiler could not compile code that takes a function containing such a state as an argument because the tree of all internal states was statically determined at compile time, but the VM in the \lambdammm\ can manage it dynamically. Listing \ref{lst:bytecode_filterbank} shows translated VM instruction of the code. Recursive calls of the first line of code in \texttt{filterbank} and the functions given as arguments or obtained via upvalue like \texttt{filter} are called with the \texttt{CALLCLS} instruction instead of the \texttt{CALL} instruction. The \texttt{GETSTATE} and \texttt{SETSTATE} instructions are not used in this function because the internal state storage is switched when the \texttt{CALLCLS} is interpreted. + + \begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_good,language=Rust,caption=\it Example code that duplicates filter parametrically using a recursive function and closure.] + fn filterbank(n,filter){ + let next = filterbank(n-1,filter) + if (n>0){ + |x,freq| filter(x,freq+n*100) + + next(x,freq) + }else{ + |x,freq| 0 + } + } + let myfilter = filterbank(3,bandpass) + fn dsp(){ + myfilter(x,1000) + } + \end{lstlisting} + + \begin{lstlisting}[float,floatplacement=H,label=lst:bytecode_filterbank,caption=\it Compiled VM instructions filterbank example in Listing \ref{lst:filterbank_good}] + CONSTANTS[100,1,0,2] + fn inner_then(x,freq) + //upvalue:[local(4),local(3),local(2),local(1)] + GETUPVALUE 3 2 //load filter + MOVE 4 0 + MOVE 5 1 + GETUPVALUE 6 1 //load n + ADDD 5 5 6 + MOVECONST 6 0 + MULF 5 5 6 + CALLCLS 3 2 1 //call filter + GETUPVALUE 4 4 //load next + MOVE 5 0 + MOVE 6 1 + CALLCLS 4 2 1 //call next + ADDF 3 3 4 + RETURN 3 1 + + fn inner_else(x,freq) + MOVECONST 2 2 + RETURN 2 1 + + fn filterbank(n,filter) + MOVECONST 2 1 //load itself + MOVE 3 0 //load n + MOVECONST 4 1 //load 1 + SUBF 3 3 4 + MOVECONST 4 2 //load inner_then + CALLCLS 2 2 1 //recursive call + MOVE 3 0 + MOVECONST 4 2 //load 0 + SUBF 3 3 4 + JMPIFNEG 3 2 + MOVECONST 3 2 //load inner_then + CLOSURE 3 3 //load inner_lambda + JMP 2 + MOVECONST 3 3 //load inner_else + CLOSURE 3 3 + CLOSE 2 + RETURN 3 1 + \end{lstlisting} + + + \section{Discussion} + \label{sec:discussion} + + As seen in the example of the filterbank, in \lambdammm, signal graph can be parametrically generated in an evaluation of global context, compared to that Faust uses a term-rewriting macro and Kronos uses a type-level computation as in the Table \ref{tab:comparison}. + + The ability to describe both the generation of parametric signal processing and its content in a single semantics will make it easier for novice users to understand the mechanism of the language. Also, the single semantics may facilitate run-time interoperation with other general-purpose languages. + + On the other hand, there is the problem that the single semantics causes \lambdammm\ to behave differently from the behavior expected in a normal lambda calculus. + + \begin{table}[ht] + \centering + \begin{tabular*}{\textwidth}{|c|c|c|}\cline{1-3} + \ & Parametric Signal Graph & Actual DSP \\\cline{1-3} + Faust & Term Rewriting Macro & BDA \\\cline{1-3} + Kronos & Type-level Computation & Value Evaluation\\\cline{1-3} + \lambdammm& \begin{tabular}{ll}Evaluation in \\ Global Context\end{tabular} & \begin{tabular}{ll}Evaluation of \\\texttt{dsp} Function\end{tabular} \\\cline{1-3} + \end{tabular*} + \caption{\label{tab:comparison}{\it Comparison of the way of signal graph generation and actual signal processing between Faust, Kronos and \lambdammm.}} + \end{table} + + \subsection{Different Behaviour Depending on the Location of Let Binding} + \label{sec:letbinding} + + By having functions that have internal states which change over time in mimium, when higher-order functions are used, there is a counterintuitive behavior compared to general functional programming languages. + + Listing \ref{lst:filterbank_bad} is an example of the incorrect code slightly modified from the filterbank example in Listing \ref{lst:filterbank_good}. The difference between Listing \ref{lst:filterbank_good} and Listing \ref{lst:filterbank_bad} is that the recursive calls in the filterbank function are written directly, or once bound with \texttt{let} expression out of the inner function. Similarly, in the \texttt{dsp} function that will be called by the audio driver in mimium, the difference is whether the filterbank function is executed inside \texttt{dsp} or bound with \texttt{let} once in the global context. + + In the case of normal functional language, if all the functions used in a composition do not contain destructive assignments, the calculation process will not change even if the variable bound by \texttt{let} were manually replaced with its term (beta reduction), as in the conversion from Listing \ref{lst:filterbank_good} to Listing \ref{lst:filterbank_bad}. + + But in mimium, there are two major stages of evaluation, 0: the code is evaluated in the global environment (concretizing the signal processing graph) at first, and 1: the dsp function is repeatedly executed (actual signal processing) and the function may involve implicit internal state updates. Therefore, even though the code does not include destructive assignments, the recursive execution of the \texttt{filterbank} function is performed only once in Listing \ref{lst:filterbank_good} for the evaluation of the global environment, whereas in Listing \ref{lst:filterbank_bad}, every sample the dsp function is executed, the recursive function is executed and a closure is generated. Since the initialization of the internal state in the closure is performed at the time of closure allocation, in the example of Listing\ref{lst:filterbank_bad}, the internal state of the closure after the evaluation of \texttt{filterbank} is reset at each time step. + + \begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_bad,language=Rust,caption=\it Wrong example of the code that duplicate filter parametrically.] + + fn bandpass(x,freq){ + //... + } + fn filterbank(n,filter){ + if (n>0){ + |x,freq| filter(x,freq+n*100) + + filterbank(n-1,filter)(x,freq) + }else{ + |x,freq| 0 + } + } + fn dsp(){ //called by audio driver. + filterbank(3,bandpass) + } + \end{lstlisting} + + + \begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_multi,language=Rust,caption=\it Example of filterbank function using multi-stage computation in a future specification of mimium.] + fn filterbank(n,filter){ + .< if (n>0){ + |x,freq| filter(x,freq+n*100) + + ~filterbank(n-1,filter)(x,freq) + }else{ + |x,freq| 0 + } >. + } + fn dsp(){ + ~filterbank(3,bandpass) (x,1000) + } + \end{lstlisting} + + This means that the major compiler optimization techniques such as the constant folding and the function inlining can not simply be appropriated for mimium. Those optimizations should be done after the evaluation of a global context and before evaluating \texttt{dsp} function. + + To solve this situation, introducing distinction whether the term should be used in global context evaluation (stage 0) and in the actual signal processing (stage 1) in type system. This can be realized with Multi-Stage Computation\cite{Taha1997}. Listing \ref{lst:filterbank_multi} is the example of \texttt{filterbank} code using BER MetaOCaml's syntaxes \texttt{..} which will generate evaluated program to be used in a next stage, and \texttt{\textasciitilde term} which embed terms evaluated at the previous stage\cite{kiselyov2014a}. + + \texttt{filterbank} function is evaluated in stage 0 while embedding itself by using \texttt{\textasciitilde}. This multi-stage computation code still has a same semantics in a generative signal graph generation and execution of the signal processing, in contrast to that Faust and Kronos. + + \subsection{A Possibility of the Foreign Stateful Function Call} + + The data structure of closure in \lambdammm\ is a combination of functions and internal states, as shown in Figure 3. The fact that\\ \texttt{filterbank} samples do not require any special handling of internal states also means that external signal processor (Unit Generator: UGen) such as oscillators and filters written in C or C++, for example, can be called from mimium in the same way as normal closure calls, and it is even possible to parametrically duplicate and combine external UGens. This is an advantage that is difficult to implement in Faust and other similar languages, but easy to implement on \lambdammm\ paradigm. + + However currently, mimium is based on sample-by-sample processing and cannot handle buffer-by-buffer value passing. Since most native unit generators perform processing on a buffer-by-buffer basis, there are not many cases where external UGens are utilized in practice for now. However, in the \lambdammm, only $feed$ terms need to be processed sample-by-sample, so it is possible to distinguish functions that can only process one sample at a time from functions that can process concurrently at the type level. As the Multi-rate specification is being considered in Faust\cite{jouvelotDependentVectorTypes2011}, it may be possible to read/write buffer between an external Unit Generator by having the compiler automatically determine the parts that can be processed as buffer-by-buffer. + + \section{Conclusion} + \label{sec:conclusion} + + This paper proposed \lambdammm , an intermediate representation for the programming languages for music and signal processing with the virtual machine and instruction set to run it. \lambdammm\ enables to describe generative signal graph and its contents in a single syntax and semantics. However, user have to be responsible to write codes that does not create escapable closures during the iterative execution of DSP, which will be difficult to understand for novice users. + + In this paper, the translation from \lambdammm\ terms from VM instructions is explained by just showing examples of the code and its expected result of instructions as well as the semantics of VM is presented with pseudo-code of the behaviour. More formal semantics and translation process should be considered along with an introduction of the multi-stage computation. + + I hope that this research will lead to more general representations of music and sound on the digital computer and more connections between the theory of languages for music and more general programming language theory. + + \section{Acknowledgments} + + This work was supported by JSPS KAKENHI (Grant No. \\JP19K21615). Also great thanks for many anonymous reviewers. + \ No newline at end of file diff --git a/src/convert_to_md.sh b/src/convert_to_md.sh new file mode 100644 index 0000000..d47d814 --- /dev/null +++ b/src/convert_to_md.sh @@ -0,0 +1,2 @@ +#!/bin/bash +pandoc -s --from=latex+smart content.tex -t markdown+hard_line_breaks+smart -o main.md \ No newline at end of file diff --git a/src/main.tex b/src/main.tex index 45ec828..d648a19 100644 --- a/src/main.tex +++ b/src/main.tex @@ -183,440 +183,7 @@ \maketitle -\begin{abstract} -This paper proposes \lambdammm , a call-by-value, simply-typed lambda calculus-based intermediate representation for a music programming language, which handles synchronous signal processing, Additionally, it introduces a virtual machine and instruction set to execute \lambdammm . Digital signal processing is represented through a syntax that incorporates the internal states of delay and feedback into the lambda calculus. \lambdammm\ extends the lambda calculus, allowing users to construct generative signal processing graphs and execute them with consistent semantics. However, a challenge arises when handling higher-order functions, as users must determine whether execution occurs within the global environment or during DSP execution. This issue can potentially be resolved through multi-stage computation. -\end{abstract} - -\section{Introduction} -\label{sec:intro} - - -Many programming languages for sound and music have been developed, but only a few possess strongly formalized semantics. One language that is both rigorously formalized and practical is Faust \cite{Orlarey2004}, which combines blocks with inputs and outputs with five primitive operations: parallel, sequential, split, merge, and recursive connection. By providing basic arithmetic, conditionals, and delays as primitive blocks, almost any type of signal processing can be written in Faust. In a later extension, a macro based on a term rewriting system was introduced, allowing users to parameterize blocks with an arbitrary number of inputs and outputs \cite{graf2010}. - -This strong abstraction capability through formalization enables Faust to be translated to various backends, such as C, C++, Rust, and LLVM IR. On the other hand, Faust's Block Diagram Algebra (BDA) lacks theoretical and practical compatibility with common programming languages. Although it is possible to call external C functions in Faust, those functions are assumed to be pure functions that do not require heap memory allocation or deallocation. Therefore, while it is easy to embed Faust in another language, it is not easy to call another language from Faust. - -In addition, a macro for Faust is an independent term rewriting system that generates BDA based on pattern matching. As a result, the arguments for pattern matching are implicitly required to be integers, which can sometimes lead to compile-time errors, despite the fact that BDA does not distinguish between real and integer types. These implicit typing rules are not intuitive for novice users. - -Proposing a computational model for signal processing based on more generic computational models, such as lambda calculus, has the potential to enable interoperability between many different general-purpose languages, and also facilitate the appropriation of existing optimization methods and the implementation of compilers and run-time. - -Currently, it has been demonstrated that BDA can be converted to a general-purpose functional language using an arrow, a higher-level abstraction of monads \cite{gaster2018}. However, higher-order functions in general-purpose functional languages are often implemented based on dynamic memory allocation and deallocation, making them difficult to use in host languages designed for real-time signal processing. - -Additionally, Kronos \cite{norilo2015} and W-calculus \cite{arias2021} are examples of lambda calculus-based abstractions influenced by Faust. Kronos is based on the theoretical foundation of System-$F\omega$, a variation of lambda calculus in which types themselves can be abstracted (i.e., a function that takes a type as input and returns a new type can be defined). In Kronos, type calculations correspond to signal graph generation, while value calculations correspond to the actual processing. Delay is the only special primitive operation in Kronos, and feedback routing can be represented as a recursive function application in type calculations. - -W-calculus includes feedback as a primitive operation, along with the ability to access the value of a variable from the past (i.e., delay). W-calculus restricts systems to those that can represent linear-time-invariant processes, such as filters and reverberators, and it defines more formal semantics, aiming for automatic proofs of linearity and the identity of graph topologies. - -Previously, the author designed the music programming language \textit{mimium} \cite{matsuura2021a}. By incorporating basic operations such as delay and feedback into lambda calculus, signal processing can be concisely expressed while maintaining a syntax similar to that of general-purpose programming languages. Notably, \textit{mimium}'s syntax is designed to resemble the Rust programming language. - -One of the earlier issues with \textit{mimium} was its inability to compile code that contained combinations of recursive or higher-order functions with stateful functions involving delay or feedback, as the compiler could not determine the data size of the internal state used in signal processing. - -In this paper, I propose the syntax and semantics of \lambdammm, an extended call-by-value simply-typed lambda calculus, as a computational model intended to serve as an intermediate representation for \textit{mimium}. Additionally, I propose a virtual machine and its instruction set, based on Lua's VM, to execute this computational model in practice. Finally, I discuss both the challenges and potential of the current \lambdammm\ model: one challenge is that users must differentiate whether a calculation occurs in a global context or during actual signal processing; another is that run-time interoperability with other programming languages could be easier than in existing DSP languages. - - -\section{Syntax} -\label{sec:syntax} - -\input{syntax.tex} - -The types and terms of \lambdammm\ are defined in Figure \ref{fig:syntax_v}. - -Two terms are introduced in addition to the standard simply-typed lambda calculus: $delay\ n\ e_1\ e_2$, which refers to the previous value of $e_1$ by $e_2$ samples (with a maximum delay of $n$ to limit memory usage to a finite size), and $feed\ x.e$, an abstraction that allows the user to refer to the result of evaluating $e$ from one time unit earlier as $x$ during the evaluation of $e$ itself. - -\subsection{Syntactic Sugar of the Feedback Expression in \textit{mimium}} -\label{sec:mimium} - -The programming language \textit{mimium}, developed by the author, includes a keyword $self$ that can be used in function definitions to refer to the previous return value of the function. An example of a simple one-pole filter function, which mixes the input and the last output signal such that the sum of the input and feedback gains is 1, is shown in Listing \ref{lst:onepole}. This code can be expressed in \lambdammm\ as shown in Figure \ref{fig:onepole}. - -\begin{lstlisting}[float,floatplacement=H,label=lst:onepole,language=Rust,caption=\it Example of the code of one-pole filter in mimium.] - fn onepole(x,g){ - x*(1.0-g) + self*g - } -\end{lstlisting} - -\begin{figure}[ht] -\begin{equation*} - \centering - \begin{aligned} - let\ & onepole = \\ - & \ \lambda x. \lambda g.\ feed\ y.\ x *(1.0 - g) + y * g \ in\ ... - \end{aligned} -\end{equation*} -\caption{\label{fig:onepole}{\it Equivalent expression to Listing \ref{lst:onepole} in $\lambda_{mmm}$.}} -\end{figure} - -\subsection{Typing Rules} -\label{sec:typing} - -\input{typing.tex} - -Additional typing rules for the usual simply-typed lambda calculus are shown in Figure \ref{fig:typing}. - -The primitive types include a real number type, used in most signal processing, and a natural number type, which is used for the indices of delay. - -In W-calculus, which directly inspired the design of \lambdammm, function types can only take tuples of real numbers and return tuples of real numbers. This restriction prevents the definition of higher-order functions. While this limitation is reasonable for a signal processing language—since higher-order functions require data structures like closures that depend on dynamic memory allocation—it also reduces the generality of the lambda calculus. - -In \lambdammm, the problem of memory allocation for closures is delegated to the runtime implementation (see Section \ref{sec:vm}), allowing the use of higher-order functions. However, the $feed$ abstraction does not permit function types as either input or output. Allowing function types in the $feed$ abstraction would enable the definition of functions whose behavior could change over time. While this is theoretically interesting, there are no practical examples in real-world signal processing, and such a feature would likely complicate implementations further. - -\section{Semantics} -\label{sec:semantics} - -\input{semantics.tex} - -The excerpt of operational semantics of the \lambdammm\ is shown in Figure \ref{fig:semantics}. This big-step semantics is a conceptual explanation of the evaluation that, when the current time is $n$, the previous evaluation environment $t$ samples before can be referred to as $E^{n-t}$ , and that when the time < 0, the evaluation of any term is evaluated to the default value of its type (0 for the numeric types). - -Of course, if we tried to execute this semantics in a straightforward manner, we would have to redo the calculation from time 0 to the current time every sample, with saving all the variable environments at each sample. In practice, therefore, a virtual machine is defined that takes into account the internal memory space used by $delay$ and $feed$, and the \lambdammm\ terms are converted into instructions for that machine before execution. - -\section{VM Model and Instruction Set} -\label{sec:vm} - -A model for the virtual machine and its instruction set to run \\ \lambdammm\ is based on the VM for Lua version 5\cite{ierusalimschy2005}. - -When executing a computational model based on lambda calculus, the problem is how to handle a data structure called a closure that captures the variable environment where the inner function is defined, to refer the outer variables from the inner function context. If the dictionary data of names and values of variables are paired with inner function, implementation of the compiler (intepreter) is simple, but run-time performance is limited. - -On the contrary, a runtime performance can be improved by performing a process called closure transformation (or lambda lifting), which analyses all the names of outer variables referred by the inner function and transforms the inner function by adding argument so that the variables can be referred explicitly, but the compiler implementation of the transformation is relatively complex. - -The Lua VM takes an intermediate approach between these two by adding the VM instructions \texttt{GETUPVALUE} / \\ \texttt{SETUPVALUE}, which allows the outer variables to be referred dynamically at runtime. The implementation of compiler and VM using \textit{upvalue} is simpler than closure conversion, while at the same time preventing execution performance degradation, as outer variables can be referred via the call stack rather than on the heap memory unless the closure object escapes from the context of the original function\cite{nystrom2021}. - -Also, upvalue helps interoperations between other programming languages, as Lua can be easily embedded through C language API and when implementing external libraries in C, programmer can access to upvalues of Lua Runtime not only the stack values in C API. - -\subsection{Instruction Set} -\label{sec:instruction} - -VM Instructions for \lambdammm\ differs from the Lua VM in the following respects. - -\begin{enumerate} - -\item{Since mimium is a statically typed language unlike Lua, instructions for basic arithmetics are provided for each type.} -\item{The call operation is separated into the normal function call and the call of closure due to its static typing similarly, and also to handle higher-order statefull functions(See \ref{sec:vmstructure} for details). } -\item{If statements are realised by a combination of two instructions, \texttt{JMP} and \texttt{JMPIFNEG}, whereas the Lua VM uses a dedicated \texttt{TEST} instructions.} -\item{Instructions related to for loop, the \texttt{SELF} instruction used for object-oriented programming and the \texttt{TABLE}-related instructions for metadata references to variables are omitted in mimium as they are not used.} -\item{Instructions related to list-related data structures are also omitted in this paper, as the implementation of data structures such as tuples and arrays was omitted in the description of the \lambdammm\ in this paper.} - -\end{enumerate} - -Instructions in \lambdammm\ VM are 32bit data with operation tag and 3 operands. Currently, a bit width for the tag and each operands are all 8 bit\footnote[1]{Reason for this is that it is easy to implemented on \texttt{enum} data structure on Rust, a host language of the latest mimium compiler. Operands bitwidth and alignment may be changed in the future.}. - -The VM of \lambdammm\ is a register machine like the Lua VM (after version 5), although the VM has no real register but the register number simply means the offset index of the call stack from the base pointer at the point of execution of the VM. The first operand of most instructions is the register number in which to store the result of the operation. - -The list of instructions is shown in Figure \ref{fig:instruction} (basic arithmetic operations are partly omitted). The notation for the instruction follows the Lua VM paper \cite[p.13]{ierusalimschy2005}. From left to right, the name of operation, a list of operands, and pseudo-code of the operation. When using each of the three operands as unsigned 8 bits, they are denoted as \texttt{A B C}. When used with a signed integer, prefix \texttt{s} is added, and when the two operand fields are used as one 16 bits, an suffix \texttt{x} is added. For example, when B and C are merged and treated as signed 16 bits, they are denoted as \texttt{sBx}. - -In pseudo-code describing an functionality, \texttt{R(A)} means that data is moved in and out through the register (call stack) at the point of base pointer for current function + \texttt{A}. \texttt{K(A)} means that it retrieves the \texttt{A}-th number in the static variable field of the compiled program. \texttt{U(A)} means that referring \texttt{A}-th upvalue of the current function. - -In addition to Lua's Upvalue operation, 4 operations related to internal state variables over time, \texttt{GETSTATE}, \texttt{SETSTATE}, \\ \texttt{SHIFTSTATE} and \texttt{DELAY} are added to compile $delay$ and $feed$ expressions. - -\begin{figure*}[ht] -\tt -\small -\begin{tabular}{lll} -MOVE & A B & R(A) := R(B) \\ -MOVECONST & A B & R(A) := K(B) \\ -GETUPVALUE & A B & R(A) := U(B) \\ -\multicolumn{3}{l}{ - \textit{(SETUPVALUE does not exist)} -}\\ -GETSTATE* & A & R(A) := SPtr[SPos] \\ -SETSTATE* & A & SPtr[SPos] := R(A) \\ -SHIFTSTATE* & sAx & SPos += sAx \\ -DELAY* & A B C & R(A) := update\_ringbuffer(SPtr[SPos],R(B),R(C)) \\ -\multicolumn{3}{l}{ - \textit{ *((SPos,SPtr)= vm.closures[vm.statepos\_stack.top()].state } -}\\ -\multicolumn{3}{l}{ - \textit{\quad if vm.statepos\_stack is empty, use global state storage.)} -}\\ -JMP & sAx & PC +=sAx \\ -JMPIFNEG & A sBx & if (R(A)<0) then PC += sBx \\ -CALL & A B C & R(A),...,R(A+C-2) := program.functions[R(A)](R(A+1),...,R(A+B-1)) \\ -CALLCLS & A B C & vm.statepos\_stack.push(R(A)) \\ -\ & \ & R(A),...,R(A+C-2) := vm.closures[R(A)].fnproto(R(A+1),...,R(A+B-1)) \\ -\ & \ & vm.statepos\_stack.pop() \\ -CLOSURE & A Bx & vm.closures.push(closure(program.functions[R(Bx)])) \\ - & & R(A) := vm.closures.length - 1 \\ -CLOSE & A & close stack variables up to R(A)\\ -RETURN & A B & return R(A), R(A+1)...,R(A+B-2) \\ -ADDF & A B C & R(A) := R(B) as float + R(C) as float\\ -SUBF & A B C & R(A) := R(B) as float - R(C) as float\\ -MULF & A B C & R(A) := R(B) as float * R(C) as float\\ -DIVF & A B C & R(A) := R(B) as float / R(C) as float\\ -ADDI & A B C & R(A) := R(B) as int + R(C) as int \\ -\ & -\multicolumn{2}{l}{ -\textit{...Other basic arithmetics continues for each primitive types...} -} -\end{tabular} -\caption{\label{fig:instruction}{\it Instruction sets for VM to run $\lambda_{mmm}$.}} -\end{figure*} - -\subsection{Overview of the VM Structure} -\label{sec:vmstructure} - -The overview of a data structure of the virtual machine, the program and the instantiated closure for \lambdammm\ is shown in Figure \ref{fig:vmstructure}. In addition to the normal call stack, the VM has a storage area for managing internal state data for feedback and delay. - -This storage area is accompanied by data indicating the position from which the internal state is retrieved by the \texttt{GETSTATE} / \texttt{SETSTATE} instructions. This position is modified by \\ \texttt{SHIFTSTATE} operation back and forth. The actual data in the state storage memory are statically layed out at compile time by analyzing function calls that include references to \texttt{self}, call of \texttt{delay} and the functions which will call such statefull functions recursively. \texttt{DELAY} operation takes 2 inputs, B for an input and C for the delay time in samples. - -However, in the case of higher-order functions that receive a function as an argument and return another function, the layout of the internal state of the given function is unknown at the compilation, so an internal state storage area is created for each instantiated closure separately from the global storage area held by the VM instance itself. The VM have an another stack to keep the pointer to state storage. Each time \texttt{CALLCLS} used, VM pushes a pointer to the state storage of instantiated closure to the state stack and, at the end of the closure call, VM pops out the state pointer from the stack. - -Instantiated closures also hold the storage area of upvalues. Until the closure exits the context of parent function (such a closure is called ``Open Closure''), upvalues holds a negative offset on the stack at the ongoing execution. This offset value can be determined at compile time, the offset is stored in the function prototype in the program. Also, not only local variables, upvalue may refer to parent funtion's upvalue (this situation can happens when at least 3 functions are nested). So the array of upvalue indexes in the function prototype holds a pair of tag whether it is local stack value or further upvalue and its index (negative offset of stack or parent function's upvalue index). - -For instance, if the Upvalue indexes in the program were like \texttt{[upvalue(1),local(3)]}, \texttt{GETUPVALUE 6 1} means that, take \texttt{3} from the upvalue indexes 1 and get value from \texttt{R(-3)} over the base pointer and store it to \texttt{R(6)}. - -When the closure escapes from the original function with \\ \texttt{RETURN} instruction, inserted \texttt{CLOSE} instruction \\ the \texttt{RETURN} instruction moves actual upvalues from the stack into somewhere on the heap memory. This upvalues may be referred from multiple locations when using nested closures, and some form of garbage collection needed to free memory after they are no longer referred. - -In the current specification, the paradigm is call-by-value and reassignment expression does not exist, therefore, \texttt{SETUPVALUE} instruction does not exist in \lambdammm\ VM. This difference also make a difference to the implemention of open upvalue in the closure because the open upvalue should be shared memory cell which maybe recursively converted into memory cell of closed value when the \texttt{CLOSE} instruction is called. - -\begin{figure*}[ht] - \centerline{\includegraphics[width=\hsize]{lambdammm_vm_structure}} - \caption{\label{fig:vmstructure}{\it Overview of the virtual machine, program and instantiated closures for \lambdammm.}} -\end{figure*} - -\subsection{Compilation to the VM Instructions} - -\begin{lstlisting}[float,floatplacement=H,label=lst:bytecodes_onepole,caption=\it Compiled VM instructions of one-pole filter example in Listing \ref{lst:onepole}] -CONSTANTS:[1.0] -fn onepole(x,g) state_size:1 -MOVECONST 2 0 // load 1.0 -MOVE 3 1 // load g -SUBF 2 2 3 // 1.0 - g -MOVE 3 0 // load x -MULF 2 2 3 // x * (1.0-g) -GETSTATE 3 // load self -MOVE 4 1 // load g -MULF 3 3 4 // self * g -ADDF 2 2 3 // compute result -GETSTATE 3 // prepare return value -SETSTATE 2 // store to self -RETURN 3 1 -\end{lstlisting} - -Listing \ref{lst:bytecodes_onepole} shows an basic example when the mimium code in Listing \ref{lst:onepole} is compiled into VM bytecode. When \texttt{self} is referred, the value is obtained with the \texttt{GETSTATE} instruction, and the internal state is updated by storing the return value with the \\ \texttt{SETSTATE} instruction before returning the value with \texttt{RETURN} from the function. Here, the actual return value is obtained by the second \texttt{GETSTATE} instruction in order to return the initial value of the internal state when time=0. - -For example, when a time counter is written as \texttt{| | \{self + 1\}}, it is the compiler's design choice whether the return value of time=0 should be 0 or 1 though the latter does not strictly follow the semantics E-FEED in Figure \ref{fig:semantics}. If the design is to return 1 when time = 0, the second \texttt{GETSTATE} instruction can be removed and the value for the \texttt{RETURN} instruction should be \texttt{R(2)}. - -A more complex example code and its expected bytecode instructions are shown in Listing \ref{lst:fbdelay} and Listing \ref{lst:bytecodes_fbdelay}. The codes define delay with a feedback as \texttt{fbdelay}, the other function \texttt{twodelay} uses two feedback delay with different parameters, and \texttt{dsp} finally uses two \texttt{twodelay} function. - -Each after the referring to \texttt{self} through \texttt{GETSTATE} instruction, or call to the other statefull function, \\ \texttt{SHIFTSTATE} instruction inserted to move the position of state storage forward to prepare the next non-closure function call. Before exiting function, the state position is reset to the same position as that the current function context has begun by \texttt{SHIFTSTATE} (A sum of the operand for \texttt{SHIFTSTATE} in a function must be always 0). Figure \ref{fig:fbdelay_spos} shows how the state position moves by \texttt{SHIFT-}\\\texttt{STATE} operations during the execution of \texttt{twodelay} function. - -By describing an internal state as a relative position in the state storage, the state data can be expressed as a flat array, which makes the implementation of the compiler simple, not like a tree structure that need to analyze a call tree from the root to generate as in the previous implementation of mimium. This is similar to upvalue makes the implementation of the compiler simpler by describing free variables as relative positions on the call stack. - -\begin{lstlisting}[float,floatplacement=H,label=lst:fbdelay,language=Rust,caption=\it Example code that combines self and delay without closure call.] -fn fbdelay(x,fb,dtime){ - x + delay(1000,self,dtime)*fb -} -fn twodelay(x,dtime){ - fbdelay(x,dtime,0.7) - +fbdelay(x,dtime*2,0.8) -} -fn dsp(x){ - twodelay(x,400)+twodelay(x,800) -} -\end{lstlisting} - -\begin{lstlisting}[float,floatplacement=H,label=lst:bytecodes_fbdelay,caption=\it Compiled VM instructions of feedback delay example in Listing \ref{lst:fbdelay}] -CONSTANTS:[0.7,2,0.8,400,800,0,1] -fn fbdelay(x,fb,dtime) state_size:2 -MOVE 3 0 //load x -GETSTATE 4 //load self -SHIFTSTATE 1 //shift Spos -DELAY 4 4 2 //delay(_,_,_) -MOVE 5 1 // load fb -MULF 4 4 5 //delayed val *fb -ADDF 3 3 4 // x+ -SHIFTSTATE -1 //reset SPos -GETSTATE 4 //prepare result -SETSTATE 3 //store to self -RETURN 4 1 //return previous self - -fn twodelay(x,dtime) state_size:4 -MOVECONST 2 5 //load "fbdelay" prototype -MOVE 3 0 -MOVE 4 1 -MOVECONST 5 0 //load 0.7 -CALL 2 3 1 -SHIFTSTATE 2 //2=state_size of fbdelay -MOVECONST 3 5 //load "fbdelay" prototype -MOVE 4 0 -MOVECONST 5 1 //load 2 -MULF 4 4 5 -MOVECONST 5 0 //load 0.7 -CALL 3 3 1 -ADDF 3 3 4 -SHIFTSTATE -2 -RETURN 3 1 - -fn dsp (x) -MOVECONST 1 6 //load "twodelay" prototype -MOVE 2 0 -MOVECONST 3 3 //load 400 -CALL 1 2 1 -SHIFTSTATE 4 //4=state_size of twodelay -MOVECONST 2 6 -MOVE 2 3 //load "twodelay" prototype -MOVE 3 0 -MOVECONST 3 4 //load 400 -CALL 2 2 1 -ADD 1 1 2 -SHIFTSTATE -4 -RETURN 1 1 -\end{lstlisting} - -\begin{figure}[ht] - \centerline{\includegraphics[width=0.7\hsize]{fbdelay_spos}} - \caption{\label{fig:fbdelay_spos}{\it Image of how the state position moves while executing \texttt{twodelay} function in Listing \ref{lst:bytecodes_fbdelay}.}} -\end{figure} - - -Listing \ref{lst:filterbank_good} shows an example of a higher-order function \\ \texttt{filterbank} that takes another function \texttt{filter} that takes an input and a frequency as an argument, duplicates \texttt{n} of filter, and adds them together. Note that in the previous specification of mimium in \cite{matsuura2021a}, the binding of new variable and destructive assignment were the same syntax (\texttt{x = a}) but the syntax for the variable binding has changed to use \texttt{let} keyword. Also, because the semantics is call-by-value paradigm, reassignment syntax never be used in the current implementation. - -The previous mimium compiler could not compile code that takes a function containing such a state as an argument because the tree of all internal states was statically determined at compile time, but the VM in the \lambdammm\ can manage it dynamically. Listing \ref{lst:bytecode_filterbank} shows translated VM instruction of the code. Recursive calls of the first line of code in \texttt{filterbank} and the functions given as arguments or obtained via upvalue like \texttt{filter} are called with the \texttt{CALLCLS} instruction instead of the \texttt{CALL} instruction. The \texttt{GETSTATE} and \texttt{SETSTATE} instructions are not used in this function because the internal state storage is switched when the \texttt{CALLCLS} is interpreted. - -\begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_good,language=Rust,caption=\it Example code that duplicates filter parametrically using a recursive function and closure.] -fn filterbank(n,filter){ - let next = filterbank(n-1,filter) - if (n>0){ - |x,freq| filter(x,freq+n*100) - + next(x,freq) - }else{ - |x,freq| 0 - } -} -let myfilter = filterbank(3,bandpass) -fn dsp(){ - myfilter(x,1000) -} -\end{lstlisting} - -\begin{lstlisting}[float,floatplacement=H,label=lst:bytecode_filterbank,caption=\it Compiled VM instructions filterbank example in Listing \ref{lst:filterbank_good}] -CONSTANTS[100,1,0,2] -fn inner_then(x,freq) -//upvalue:[local(4),local(3),local(2),local(1)] -GETUPVALUE 3 2 //load filter -MOVE 4 0 -MOVE 5 1 -GETUPVALUE 6 1 //load n -ADDD 5 5 6 -MOVECONST 6 0 -MULF 5 5 6 -CALLCLS 3 2 1 //call filter -GETUPVALUE 4 4 //load next -MOVE 5 0 -MOVE 6 1 -CALLCLS 4 2 1 //call next -ADDF 3 3 4 -RETURN 3 1 - -fn inner_else(x,freq) -MOVECONST 2 2 -RETURN 2 1 - -fn filterbank(n,filter) -MOVECONST 2 1 //load itself -MOVE 3 0 //load n -MOVECONST 4 1 //load 1 -SUBF 3 3 4 -MOVECONST 4 2 //load inner_then -CALLCLS 2 2 1 //recursive call -MOVE 3 0 -MOVECONST 4 2 //load 0 -SUBF 3 3 4 -JMPIFNEG 3 2 -MOVECONST 3 2 //load inner_then -CLOSURE 3 3 //load inner_lambda -JMP 2 -MOVECONST 3 3 //load inner_else -CLOSURE 3 3 -CLOSE 2 -RETURN 3 1 -\end{lstlisting} - - -\section{Discussion} -\label{sec:discussion} - -As seen in the example of the filterbank, in \lambdammm, signal graph can be parametrically generated in an evaluation of global context, compared to that Faust uses a term-rewriting macro and Kronos uses a type-level computation as in the Table \ref{tab:comparison}. - -The ability to describe both the generation of parametric signal processing and its content in a single semantics will make it easier for novice users to understand the mechanism of the language. Also, the single semantics may facilitate run-time interoperation with other general-purpose languages. - -On the other hand, there is the problem that the single semantics causes \lambdammm\ to behave differently from the behavior expected in a normal lambda calculus. - -\begin{table}[ht] -\centering -\begin{tabular*}{\textwidth}{|c|c|c|}\cline{1-3} - \ & Parametric Signal Graph & Actual DSP \\\cline{1-3} - Faust & Term Rewriting Macro & BDA \\\cline{1-3} - Kronos & Type-level Computation & Value Evaluation\\\cline{1-3} - \lambdammm& \begin{tabular}{ll}Evaluation in \\ Global Context\end{tabular} & \begin{tabular}{ll}Evaluation of \\\texttt{dsp} Function\end{tabular} \\\cline{1-3} -\end{tabular*} -\caption{\label{tab:comparison}{\it Comparison of the way of signal graph generation and actual signal processing between Faust, Kronos and \lambdammm.}} -\end{table} - -\subsection{Different Behaviour Depending on the Location of Let Binding} -\label{sec:letbinding} - -By having functions that have internal states which change over time in mimium, when higher-order functions are used, there is a counterintuitive behavior compared to general functional programming languages. - -Listing \ref{lst:filterbank_bad} is an example of the incorrect code slightly modified from the filterbank example in Listing \ref{lst:filterbank_good}. The difference between Listing \ref{lst:filterbank_good} and Listing \ref{lst:filterbank_bad} is that the recursive calls in the filterbank function are written directly, or once bound with \texttt{let} expression out of the inner function. Similarly, in the \texttt{dsp} function that will be called by the audio driver in mimium, the difference is whether the filterbank function is executed inside \texttt{dsp} or bound with \texttt{let} once in the global context. - -In the case of normal functional language, if all the functions used in a composition do not contain destructive assignments, the calculation process will not change even if the variable bound by \texttt{let} were manually replaced with its term (beta reduction), as in the conversion from Listing \ref{lst:filterbank_good} to Listing \ref{lst:filterbank_bad}. - -But in mimium, there are two major stages of evaluation, 0: the code is evaluated in the global environment (concretizing the signal processing graph) at first, and 1: the dsp function is repeatedly executed (actual signal processing) and the function may involve implicit internal state updates. Therefore, even though the code does not include destructive assignments, the recursive execution of the \texttt{filterbank} function is performed only once in Listing \ref{lst:filterbank_good} for the evaluation of the global environment, whereas in Listing \ref{lst:filterbank_bad}, every sample the dsp function is executed, the recursive function is executed and a closure is generated. Since the initialization of the internal state in the closure is performed at the time of closure allocation, in the example of Listing\ref{lst:filterbank_bad}, the internal state of the closure after the evaluation of \texttt{filterbank} is reset at each time step. - -\begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_bad,language=Rust,caption=\it Wrong example of the code that duplicate filter parametrically.] - -fn bandpass(x,freq){ - //... -} -fn filterbank(n,filter){ - if (n>0){ - |x,freq| filter(x,freq+n*100) - + filterbank(n-1,filter)(x,freq) - }else{ - |x,freq| 0 - } -} -fn dsp(){ //called by audio driver. - filterbank(3,bandpass) -} -\end{lstlisting} - - -\begin{lstlisting}[float,floatplacement=H,label=lst:filterbank_multi,language=Rust,caption=\it Example of filterbank function using multi-stage computation in a future specification of mimium.] - fn filterbank(n,filter){ - .< if (n>0){ - |x,freq| filter(x,freq+n*100) - + ~filterbank(n-1,filter)(x,freq) - }else{ - |x,freq| 0 - } >. - } - fn dsp(){ - ~filterbank(3,bandpass) (x,1000) - } -\end{lstlisting} - -This means that the major compiler optimization techniques such as the constant folding and the function inlining can not simply be appropriated for mimium. Those optimizations should be done after the evaluation of a global context and before evaluating \texttt{dsp} function. - -To solve this situation, introducing distinction whether the term should be used in global context evaluation (stage 0) and in the actual signal processing (stage 1) in type system. This can be realized with Multi-Stage Computation\cite{Taha1997}. Listing \ref{lst:filterbank_multi} is the example of \texttt{filterbank} code using BER MetaOCaml's syntaxes \texttt{..} which will generate evaluated program to be used in a next stage, and \texttt{\textasciitilde term} which embed terms evaluated at the previous stage\cite{kiselyov2014a}. - -\texttt{filterbank} function is evaluated in stage 0 while embedding itself by using \texttt{\textasciitilde}. This multi-stage computation code still has a same semantics in a generative signal graph generation and execution of the signal processing, in contrast to that Faust and Kronos. - -\subsection{A Possibility of the Foreign Stateful Function Call} - -The data structure of closure in \lambdammm\ is a combination of functions and internal states, as shown in Figure 3. The fact that\\ \texttt{filterbank} samples do not require any special handling of internal states also means that external signal processor (Unit Generator: UGen) such as oscillators and filters written in C or C++, for example, can be called from mimium in the same way as normal closure calls, and it is even possible to parametrically duplicate and combine external UGens. This is an advantage that is difficult to implement in Faust and other similar languages, but easy to implement on \lambdammm\ paradigm. - -However currently, mimium is based on sample-by-sample processing and cannot handle buffer-by-buffer value passing. Since most native unit generators perform processing on a buffer-by-buffer basis, there are not many cases where external UGens are utilized in practice for now. However, in the \lambdammm, only $feed$ terms need to be processed sample-by-sample, so it is possible to distinguish functions that can only process one sample at a time from functions that can process concurrently at the type level. As the Multi-rate specification is being considered in Faust\cite{jouvelotDependentVectorTypes2011}, it may be possible to read/write buffer between an external Unit Generator by having the compiler automatically determine the parts that can be processed as buffer-by-buffer. - -\section{Conclusion} -\label{sec:conclusion} - -This paper proposed \lambdammm , an intermediate representation for the programming languages for music and signal processing with the virtual machine and instruction set to run it. \lambdammm\ enables to describe generative signal graph and its contents in a single syntax and semantics. However, user have to be responsible to write codes that does not create escapable closures during the iterative execution of DSP, which will be difficult to understand for novice users. - -In this paper, the translation from \lambdammm\ terms from VM instructions is explained by just showing examples of the code and its expected result of instructions as well as the semantics of VM is presented with pseudo-code of the behaviour. More formal semantics and translation process should be considered along with an introduction of the multi-stage computation. - -I hope that this research will lead to more general representations of music and sound on the digital computer and more connections between the theory of languages for music and more general programming language theory. - -\section{Acknowledgments} - -This work was supported by JSPS KAKENHI (Grant No. \\JP19K21615). Also great thanks for many anonymous reviewers. - +\input{content} %\newpage \nocite{*} \bibliographystyle{IEEEbib} diff --git a/src/main_ja.md b/src/main_ja.md new file mode 100644 index 0000000..d92d1d5 --- /dev/null +++ b/src/main_ja.md @@ -0,0 +1,480 @@ +--- +abstract: | + This paper proposes , a call-by-value, simply-typed lambda calculus-based intermediate representation for a music programming language, which handles synchronous signal processing, Additionally, it introduces a virtual machine and instruction set to execute . Digital signal processing is represented through a syntax that incorporates the internal states of delay and feedback into the lambda calculus.  extends the lambda calculus, allowing users to construct generative signal processing graphs and execute them with consistent semantics. However, a challenge arises when handling higher-order functions, as users must determine whether execution occurs within the global environment or during DSP execution. This issue can potentially be resolved through multi-stage computation. +--- + +# Introduction {#sec:intro} + +Many programming languages for sound and music have been developed, but only a few possess strongly formalized semantics. One language that is both rigorously formalized and practical is Faust [@Orlarey2004], which combines blocks with inputs and outputs with five primitive operations: parallel, sequential, split, merge, and recursive connection. By providing basic arithmetic, conditionals, and delays as primitive blocks, almost any type of signal processing can be written in Faust. In a later extension, a macro based on a term rewriting system was introduced, allowing users to parameterize blocks with an arbitrary number of inputs and outputs [@graf2010]. + +This strong abstraction capability through formalization enables Faust to be translated to various backends, such as C, C++, Rust, and LLVM IR. On the other hand, Faust's Block Diagram Algebra (BDA) lacks theoretical and practical compatibility with common programming languages. Although it is possible to call external C functions in Faust, those functions are assumed to be pure functions that do not require heap memory allocation or deallocation. Therefore, while it is easy to embed Faust in another language, it is not easy to call another language from Faust. + +In addition, a macro for Faust is an independent term rewriting system that generates BDA based on pattern matching. As a result, the arguments for pattern matching are implicitly required to be integers, which can sometimes lead to compile-time errors, despite the fact that BDA does not distinguish between real and integer types. These implicit typing rules are not intuitive for novice users. + +Proposing a computational model for signal processing based on more generic computational models, such as lambda calculus, has the potential to enable interoperability between many different general-purpose languages, and also facilitate the appropriation of existing optimization methods and the implementation of compilers and run-time. + +Currently, it has been demonstrated that BDA can be converted to a general-purpose functional language using an arrow, a higher-level abstraction of monads [@gaster2018]. However, higher-order functions in general-purpose functional languages are often implemented based on dynamic memory allocation and deallocation, making them difficult to use in host languages designed for real-time signal processing. + +Additionally, Kronos [@norilo2015] and W-calculus [@arias2021] are examples of lambda calculus-based abstractions influenced by Faust. Kronos is based on the theoretical foundation of System-$F\omega$, a variation of lambda calculus in which types themselves can be abstracted (i.e., a function that takes a type as input and returns a new type can be defined). In Kronos, type calculations correspond to signal graph generation, while value calculations correspond to the actual processing. Delay is the only special primitive operation in Kronos, and feedback routing can be represented as a recursive function application in type calculations. + +W-calculus includes feedback as a primitive operation, along with the ability to access the value of a variable from the past (i.e., delay). W-calculus restricts systems to those that can represent linear-time-invariant processes, such as filters and reverberators, and it defines more formal semantics, aiming for automatic proofs of linearity and the identity of graph topologies. + +Previously, the author designed the music programming language *mimium* [@matsuura2021a]. By incorporating basic operations such as delay and feedback into lambda calculus, signal processing can be concisely expressed while maintaining a syntax similar to that of general-purpose programming languages. Notably, *mimium*'s syntax is designed to resemble the Rust programming language. + +One of the earlier issues with *mimium* was its inability to compile code that contained combinations of recursive or higher-order functions with stateful functions involving delay or feedback, as the compiler could not determine the data size of the internal state used in signal processing. + +In this paper, I propose the syntax and semantics of , an extended call-by-value simply-typed lambda calculus, as a computational model intended to serve as an intermediate representation for *mimium*. Additionally, I propose a virtual machine and its instruction set, based on Lua's VM, to execute this computational model in practice. Finally, I discuss both the challenges and potential of the current  model: one challenge is that users must differentiate whether a calculation occurs in a global context or during actual signal processing; another is that run-time interoperability with other programming languages could be easier than in existing DSP languages. + +# Syntax {#sec:syntax} + +
+ + + + + + + + + + +
$$\begin{aligned} + \tau_p ::= &\; R\ &[real] \\ + \ | &\; N\ &[nat] \\ + \tau ::= &\; \tau_p\ & \\ + \ | &\; \tau \to \tau \ &[function] \\ + % |&\quad \langle \tau \rangle + \end{aligned}$$ Types$$\begin{aligned} + v_p \; ::=& \ r \quad r \in \mathbb{R} \\ + |& \ n \quad n \in \mathbb{N}\\ + v \; ::=&\ v_p \\ + |& \ cls(\lambda\ x.e, E) \\ + %%|& \quad (e_1,e_2) \quad & [product]\\ + %%|& \quad \pi_n e \quad n\in \mathbb{N},\; n>0 \quad & [project]\\ + %%|& \quad \langle e \rangle \quad & [code] \\ + %%|& \quad \textasciitilde e \quad & [escape] + \end{aligned}$$ Values
$$\begin{aligned} +e \; ::=& \ x &x \in {v_p} \ & [value]\\ + |& \ \lambda x.e & & [lambda]\\ + |& \ let\; x = e_1\; in\; e_2 & & [let]\\ + |& \ fix \; x.e & & [fixpoint]\\ + |& \ e_1 \; e_2 & & [app]\\ + |& \ if\; (e_c)\; e_t\; else\; e_e \; & & [if] \\ + |& \ delay\; n \; e_1 \; e_2 &n \in \mathbb{N}\ & [delay]\\ + |& \ feed \; x.e & & [feed]\\ + |& ... & & \\ + %%|& \quad (e_1,e_2) \quad & [product]\\ + %%|& \quad \pi_n e \quad n\in \mathbb{N},\; n>0 \quad & [project]\\ + %%|& \quad \langle e \rangle \quad & [code] \\ + %%|& \quad \textasciitilde e \quad & [escape] + \end{aligned}$$ Terms
+
Definition of Types, Values and Terms of the λmmm(Basic arithmetics are omitted).
+
+ +The types and terms of  are defined in Figure [1](#fig:syntax_v){reference-type="ref" reference="fig:syntax_v"}. + +Two terms are introduced in addition to the standard simply-typed lambda calculus: $delay\ n\ e_1\ e_2$, which refers to the previous value of $e_1$ by $e_2$ samples (with a maximum delay of $n$ to limit memory usage to a finite size), and $feed\ x.e$, an abstraction that allows the user to refer to the result of evaluating $e$ from one time unit earlier as $x$ during the evaluation of $e$ itself. + +## Syntactic Sugar of the Feedback Expression in *mimium* {#sec:mimium} + +The programming language *mimium*, developed by the author, includes a keyword $self$ that can be used in function definitions to refer to the previous return value of the function. An example of a simple one-pole filter function, which mixes the input and the last output signal such that the sum of the input and feedback gains is 1, is shown in Listing [\[lst:onepole\]](#lst:onepole){reference-type="ref" reference="lst:onepole"}. This code can be expressed in  as shown in Figure [2](#fig:onepole){reference-type="ref" reference="fig:onepole"}. + +``` {#lst:onepole .Rust float="" floatplacement="H" label="lst:onepole" language="Rust" caption="\\it Example of the code of one-pole filter in mimium."} +fn onepole(x,g){ + x*(1.0-g) + self*g + } +``` + +
+

$$\centering + \begin{aligned} + let\ & onepole = \\ + & \ \lambda x. \lambda g.\ feed\ y.\ x *(1.0 - g) + y * g \ in\ ... + \end{aligned}$$

+
Equivalent expression to Listing [lst:onepole] in λmmm.
+
+ +## Typing Rules {#sec:typing} + +
+ + + + + + + + + + + +
$$\frac{\Gamma, x:\tau_a \vdash e:\tau_b}{\Gamma \vdash \lambda x.e:\tau_a \to \tau_b }$$T-LAMBDA$$\frac{ \Gamma \vdash e_1:N \quad \Gamma \vdash e_2:\tau }{\Gamma \vdash delay\ e_1\ e_2 : \tau}$$T-DELAY
$$\frac{\Gamma, x : \tau_p \vdash e: \tau_p }{\Gamma \vdash feed\ x.e:\tau_p}$$T-FEED$$\frac{ \Gamma \vdash e_c : R\ \Gamma \vdash e_t:\tau\ \Gamma \vdash e_e:\tau }{\Gamma \vdash if\ (e_c)\ e_t\ e_e\ : \tau}$$T-IF
+
Excerpt of the typing rules for .
+
+ +Additional typing rules for the usual simply-typed lambda calculus are shown in Figure [3](#fig:typing){reference-type="ref" reference="fig:typing"}. + +The primitive types include a real number type, used in most signal processing, and a natural number type, which is used for the indices of delay. + +In W-calculus, which directly inspired the design of , function types can only take tuples of real numbers and return tuples of real numbers. This restriction prevents the definition of higher-order functions. While this limitation is reasonable for a signal processing language---since higher-order functions require data structures like closures that depend on dynamic memory allocation---it also reduces the generality of the lambda calculus. + +In , the problem of memory allocation for closures is delegated to the runtime implementation (see Section [4](#sec:vm){reference-type="ref" reference="sec:vm"}), allowing the use of higher-order functions. However, the $feed$ abstraction does not permit function types as either input or output. Allowing function types in the $feed$ abstraction would enable the definition of functions whose behavior could change over time. While this is theoretically interesting, there are no practical examples in real-world signal processing, and such a feature would likely complicate implementations further. + +# Semantics {#sec:semantics} + +::: figure* + ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------- + $$\frac{E^n \vdash e_2 \Downarrow v_d \ n>v_d \ E^{n-v_d} \vdash e_1 \Downarrow v}{E^n \vdash\ delay\ n\ e_1\ e_2 \Downarrow v}$$[E-DELAY]{.roman} $$\frac{}{E^n \vdash\ \lambda x.e \Downarrow cls(\lambda x.e , E^n) }$$[E-LAM]{.roman} $$\frac{ E^{n-1} \vdash e \Downarrow v_f\ E^n, x \mapsto v_f \vdash e \Downarrow v }{E^n \vdash\ feed\ x.e \Downarrow v}$$[E-FEED]{.roman} + $$\frac{E^n \vdash e_c \Downarrow n \quad n > 0\ E^n \vdash e_t\ \Downarrow v\ }{E^n \vdash\ if (e_c)\ e_t\ else\ e_e \Downarrow v }$$[E-IFTRUE]{.roman} $$\frac{E^n \vdash e_c \Downarrow n \quad n \leqq0\ E^n \vdash e_e\ \Downarrow v\ }{E^n \vdash\ if (e_c)\ e_t\ else\ e_e \Downarrow v }$$[E-IFFALSE]{.roman}   + $$\frac{E^n \vdash e_1 \Downarrow cls(\lambda x_c.e_c, E^n_c) E^n \vdash e_2 \Downarrow v_2\ E^n_c,\ x_c \mapsto v_2 \vdash e_c \Downarrow v }{E^n \vdash\ e_1\ e_2 \Downarrow v }$$[E-APP]{.roman} + ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------- +::: + +The excerpt of operational semantics of the  is shown in Figure [\[fig:semantics\]](#fig:semantics){reference-type="ref" reference="fig:semantics"}. This big-step semantics is a conceptual explanation of the evaluation that, when the current time is $n$, the previous evaluation environment $t$ samples before can be referred to as $E^{n-t}$ , and that when the time \< 0, the evaluation of any term is evaluated to the default value of its type (0 for the numeric types). + +Of course, if we tried to execute this semantics in a straightforward manner, we would have to redo the calculation from time 0 to the current time every sample, with saving all the variable environments at each sample. In practice, therefore, a virtual machine is defined that takes into account the internal memory space used by $delay$ and $feed$, and the  terms are converted into instructions for that machine before execution. + +# VM Model and Instruction Set {#sec:vm} + +A model for the virtual machine and its instruction set to run + is based on the VM for Lua version 5[@ierusalimschy2005]. + +When executing a computational model based on lambda calculus, the problem is how to handle a data structure called a closure that captures the variable environment where the inner function is defined, to refer the outer variables from the inner function context. If the dictionary data of names and values of variables are paired with inner function, implementation of the compiler (intepreter) is simple, but run-time performance is limited. + +On the contrary, a runtime performance can be improved by performing a process called closure transformation (or lambda lifting), which analyses all the names of outer variables referred by the inner function and transforms the inner function by adding argument so that the variables can be referred explicitly, but the compiler implementation of the transformation is relatively complex. + +The Lua VM takes an intermediate approach between these two by adding the VM instructions `GETUPVALUE` / +`SETUPVALUE`, which allows the outer variables to be referred dynamically at runtime. The implementation of compiler and VM using *upvalue* is simpler than closure conversion, while at the same time preventing execution performance degradation, as outer variables can be referred via the call stack rather than on the heap memory unless the closure object escapes from the context of the original function[@nystrom2021]. + +Also, upvalue helps interoperations between other programming languages, as Lua can be easily embedded through C language API and when implementing external libraries in C, programmer can access to upvalues of Lua Runtime not only the stack values in C API. + +## Instruction Set {#sec:instruction} + +VM Instructions for  differs from the Lua VM in the following respects. + +1. Since mimium is a statically typed language unlike Lua, instructions for basic arithmetics are provided for each type. + +2. The call operation is separated into the normal function call and the call of closure due to its static typing similarly, and also to handle higher-order statefull functions(See [4.2](#sec:vmstructure){reference-type="ref" reference="sec:vmstructure"} for details). + +3. If statements are realised by a combination of two instructions, `JMP` and `JMPIFNEG`, whereas the Lua VM uses a dedicated `TEST` instructions. + +4. Instructions related to for loop, the `SELF` instruction used for object-oriented programming and the `TABLE`-related instructions for metadata references to variables are omitted in mimium as they are not used. + +5. Instructions related to list-related data structures are also omitted in this paper, as the implementation of data structures such as tuples and arrays was omitted in the description of the  in this paper. + +Instructions in  VM are 32bit data with operation tag and 3 operands. Currently, a bit width for the tag and each operands are all 8 bit[^1]. + +The VM of  is a register machine like the Lua VM (after version 5), although the VM has no real register but the register number simply means the offset index of the call stack from the base pointer at the point of execution of the VM. The first operand of most instructions is the register number in which to store the result of the operation. + +The list of instructions is shown in Figure [\[fig:instruction\]](#fig:instruction){reference-type="ref" reference="fig:instruction"} (basic arithmetic operations are partly omitted). The notation for the instruction follows the Lua VM paper [@ierusalimschy2005 p.13]. From left to right, the name of operation, a list of operands, and pseudo-code of the operation. When using each of the three operands as unsigned 8 bits, they are denoted as `A B C`. When used with a signed integer, prefix `s` is added, and when the two operand fields are used as one 16 bits, an suffix `x` is added. For example, when B and C are merged and treated as signed 16 bits, they are denoted as `sBx`. + +In pseudo-code describing an functionality, `R(A)` means that data is moved in and out through the register (call stack) at the point of base pointer for current function + `A`. `K(A)` means that it retrieves the `A`-th number in the static variable field of the compiled program. `U(A)` means that referring `A`-th upvalue of the current function. + +In addition to Lua's Upvalue operation, 4 operations related to internal state variables over time, `GETSTATE`, `SETSTATE`, +`SHIFTSTATE` and `DELAY` are added to compile $delay$ and $feed$ expressions. + +::: figure* +` ` + + ---------------------------------------------------------------- ---------------------------------------------------------------------- ------------------------------------------------------------------------- + MOVE A B R(A) := R(B) + MOVECONST A B R(A) := K(B) + GETUPVALUE A B R(A) := U(B) + *(SETUPVALUE does not exist)* + GETSTATE\* A R(A) := SPtr\[SPos\] + SETSTATE\* A SPtr\[SPos\] := R(A) + SHIFTSTATE\* sAx SPos += sAx + DELAY\* A B C R(A) := update_ringbuffer(SPtr\[SPos\],R(B),R(C)) + *\*((SPos,SPtr)= vm.closures\[vm.statepos_stack.top()\].state* + *if vm.statepos_stack is empty, use global state storage.)* + JMP sAx PC +=sAx + JMPIFNEG A sBx if (R(A)\<0) then PC += sBx + CALL A B C R(A),\...,R(A+C-2) := program.functions\[R(A)\](R(A+1),\...,R(A+B-1)) + CALLCLS A B C vm.statepos_stack.push(R(A)) +     R(A),\...,R(A+C-2) := vm.closures\[R(A)\].fnproto(R(A+1),\...,R(A+B-1)) +     vm.statepos_stack.pop() + CLOSURE A Bx vm.closures.push(closure(program.functions\[R(Bx)\])) + R(A) := vm.closures.length - 1 + CLOSE A close stack variables up to R(A) + RETURN A B return R(A), R(A+1)\...,R(A+B-2) + ADDF A B C R(A) := R(B) as float + R(C) as float + SUBF A B C R(A) := R(B) as float - R(C) as float + MULF A B C R(A) := R(B) as float \* R(C) as float + DIVF A B C R(A) := R(B) as float / R(C) as float + ADDI A B C R(A) := R(B) as int + R(C) as int +   *\...Other basic arithmetics continues for each primitive types\...* + ---------------------------------------------------------------- ---------------------------------------------------------------------- ------------------------------------------------------------------------- +::: + +## Overview of the VM Structure {#sec:vmstructure} + +The overview of a data structure of the virtual machine, the program and the instantiated closure for  is shown in Figure [\[fig:vmstructure\]](#fig:vmstructure){reference-type="ref" reference="fig:vmstructure"}. In addition to the normal call stack, the VM has a storage area for managing internal state data for feedback and delay. + +This storage area is accompanied by data indicating the position from which the internal state is retrieved by the `GETSTATE` / `SETSTATE` instructions. This position is modified by +`SHIFTSTATE` operation back and forth. The actual data in the state storage memory are statically layed out at compile time by analyzing function calls that include references to `self`, call of `delay` and the functions which will call such statefull functions recursively. `DELAY` operation takes 2 inputs, B for an input and C for the delay time in samples. + +However, in the case of higher-order functions that receive a function as an argument and return another function, the layout of the internal state of the given function is unknown at the compilation, so an internal state storage area is created for each instantiated closure separately from the global storage area held by the VM instance itself. The VM have an another stack to keep the pointer to state storage. Each time `CALLCLS` used, VM pushes a pointer to the state storage of instantiated closure to the state stack and, at the end of the closure call, VM pops out the state pointer from the stack. + +Instantiated closures also hold the storage area of upvalues. Until the closure exits the context of parent function (such a closure is called "Open Closure"), upvalues holds a negative offset on the stack at the ongoing execution. This offset value can be determined at compile time, the offset is stored in the function prototype in the program. Also, not only local variables, upvalue may refer to parent funtion's upvalue (this situation can happens when at least 3 functions are nested). So the array of upvalue indexes in the function prototype holds a pair of tag whether it is local stack value or further upvalue and its index (negative offset of stack or parent function's upvalue index). + +For instance, if the Upvalue indexes in the program were like `[upvalue(1),local(3)]`, `GETUPVALUE 6 1` means that, take `3` from the upvalue indexes 1 and get value from `R(-3)` over the base pointer and store it to `R(6)`. + +When the closure escapes from the original function with +`RETURN` instruction, inserted `CLOSE` instruction +the `RETURN` instruction moves actual upvalues from the stack into somewhere on the heap memory. This upvalues may be referred from multiple locations when using nested closures, and some form of garbage collection needed to free memory after they are no longer referred. + +In the current specification, the paradigm is call-by-value and reassignment expression does not exist, therefore, `SETUPVALUE` instruction does not exist in  VM. This difference also make a difference to the implemention of open upvalue in the closure because the open upvalue should be shared memory cell which maybe recursively converted into memory cell of closed value when the `CLOSE` instruction is called. + +::: figure* +![image](lambdammm_vm_structure.pdf){width="\\hsize"} +::: + +## Compilation to the VM Instructions + +``` {#lst:bytecodes_onepole float="" floatplacement="H" label="lst:bytecodes_onepole" caption="\\it Compiled VM instructions of one-pole filter example in Listing \\ref{lst:onepole}"} +CONSTANTS:[1.0] + fn onepole(x,g) state_size:1 + MOVECONST 2 0 // load 1.0 + MOVE 3 1 // load g + SUBF 2 2 3 // 1.0 - g + MOVE 3 0 // load x + MULF 2 2 3 // x * (1.0-g) + GETSTATE 3 // load self + MOVE 4 1 // load g + MULF 3 3 4 // self * g + ADDF 2 2 3 // compute result + GETSTATE 3 // prepare return value + SETSTATE 2 // store to self + RETURN 3 1 +``` + +Listing [\[lst:bytecodes_onepole\]](#lst:bytecodes_onepole){reference-type="ref" reference="lst:bytecodes_onepole"} shows an basic example when the mimium code in Listing [\[lst:onepole\]](#lst:onepole){reference-type="ref" reference="lst:onepole"} is compiled into VM bytecode. When `self` is referred, the value is obtained with the `GETSTATE` instruction, and the internal state is updated by storing the return value with the +`SETSTATE` instruction before returning the value with `RETURN` from the function. Here, the actual return value is obtained by the second `GETSTATE` instruction in order to return the initial value of the internal state when time=0. + +For example, when a time counter is written as `| | {self + 1}`, it is the compiler's design choice whether the return value of time=0 should be 0 or 1 though the latter does not strictly follow the semantics E-FEED in Figure [\[fig:semantics\]](#fig:semantics){reference-type="ref" reference="fig:semantics"}. If the design is to return 1 when time = 0, the second `GETSTATE` instruction can be removed and the value for the `RETURN` instruction should be `R(2)`. + +A more complex example code and its expected bytecode instructions are shown in Listing [\[lst:fbdelay\]](#lst:fbdelay){reference-type="ref" reference="lst:fbdelay"} and Listing [\[lst:bytecodes_fbdelay\]](#lst:bytecodes_fbdelay){reference-type="ref" reference="lst:bytecodes_fbdelay"}. The codes define delay with a feedback as `fbdelay`, the other function `twodelay` uses two feedback delay with different parameters, and `dsp` finally uses two `twodelay` function. + +Each after the referring to `self` through `GETSTATE` instruction, or call to the other statefull function, +`SHIFTSTATE` instruction inserted to move the position of state storage forward to prepare the next non-closure function call. Before exiting function, the state position is reset to the same position as that the current function context has begun by `SHIFTSTATE` (A sum of the operand for `SHIFTSTATE` in a function must be always 0). Figure [4](#fig:fbdelay_spos){reference-type="ref" reference="fig:fbdelay_spos"} shows how the state position moves by `SHIFT-` +`STATE` operations during the execution of `twodelay` function. + +By describing an internal state as a relative position in the state storage, the state data can be expressed as a flat array, which makes the implementation of the compiler simple, not like a tree structure that need to analyze a call tree from the root to generate as in the previous implementation of mimium. This is similar to upvalue makes the implementation of the compiler simpler by describing free variables as relative positions on the call stack. + +``` {#lst:fbdelay .Rust float="" floatplacement="H" label="lst:fbdelay" language="Rust" caption="\\it Example code that combines self and delay without closure call."} +fn fbdelay(x,fb,dtime){ + x + delay(1000,self,dtime)*fb + } + fn twodelay(x,dtime){ + fbdelay(x,dtime,0.7) + +fbdelay(x,dtime*2,0.8) + } + fn dsp(x){ + twodelay(x,400)+twodelay(x,800) + } +``` + +``` {#lst:bytecodes_fbdelay float="" floatplacement="H" label="lst:bytecodes_fbdelay" caption="\\it Compiled VM instructions of feedback delay example in Listing \\ref{lst:fbdelay}"} +CONSTANTS:[0.7,2,0.8,400,800,0,1] + fn fbdelay(x,fb,dtime) state_size:2 + MOVE 3 0 //load x + GETSTATE 4 //load self + SHIFTSTATE 1 //shift Spos + DELAY 4 4 2 //delay(_,_,_) + MOVE 5 1 // load fb + MULF 4 4 5 //delayed val *fb + ADDF 3 3 4 // x+ + SHIFTSTATE -1 //reset SPos + GETSTATE 4 //prepare result + SETSTATE 3 //store to self + RETURN 4 1 //return previous self + + fn twodelay(x,dtime) state_size:4 + MOVECONST 2 5 //load "fbdelay" prototype + MOVE 3 0 + MOVE 4 1 + MOVECONST 5 0 //load 0.7 + CALL 2 3 1 + SHIFTSTATE 2 //2=state_size of fbdelay + MOVECONST 3 5 //load "fbdelay" prototype + MOVE 4 0 + MOVECONST 5 1 //load 2 + MULF 4 4 5 + MOVECONST 5 0 //load 0.7 + CALL 3 3 1 + ADDF 3 3 4 + SHIFTSTATE -2 + RETURN 3 1 + + fn dsp (x) + MOVECONST 1 6 //load "twodelay" prototype + MOVE 2 0 + MOVECONST 3 3 //load 400 + CALL 1 2 1 + SHIFTSTATE 4 //4=state_size of twodelay + MOVECONST 2 6 + MOVE 2 3 //load "twodelay" prototype + MOVE 3 0 + MOVECONST 3 4 //load 400 + CALL 2 2 1 + ADD 1 1 2 + SHIFTSTATE -4 + RETURN 1 1 +``` + +![[]{#fig:fbdelay_spos label="fig:fbdelay_spos"}*Image of how the state position moves while executing `twodelay` function in Listing [\[lst:bytecodes_fbdelay\]](#lst:bytecodes_fbdelay){reference-type="ref" reference="lst:bytecodes_fbdelay"}.*](fbdelay_spos.pdf){#fig:fbdelay_spos width="0.7\\hsize"} + +Listing [\[lst:filterbank_good\]](#lst:filterbank_good){reference-type="ref" reference="lst:filterbank_good"} shows an example of a higher-order function +`filterbank` that takes another function `filter` that takes an input and a frequency as an argument, duplicates `n` of filter, and adds them together. Note that in the previous specification of mimium in [@matsuura2021a], the binding of new variable and destructive assignment were the same syntax (`x = a`) but the syntax for the variable binding has changed to use `let` keyword. Also, because the semantics is call-by-value paradigm, reassignment syntax never be used in the current implementation. + +The previous mimium compiler could not compile code that takes a function containing such a state as an argument because the tree of all internal states was statically determined at compile time, but the VM in the  can manage it dynamically. Listing [\[lst:bytecode_filterbank\]](#lst:bytecode_filterbank){reference-type="ref" reference="lst:bytecode_filterbank"} shows translated VM instruction of the code. Recursive calls of the first line of code in `filterbank` and the functions given as arguments or obtained via upvalue like `filter` are called with the `CALLCLS` instruction instead of the `CALL` instruction. The `GETSTATE` and `SETSTATE` instructions are not used in this function because the internal state storage is switched when the `CALLCLS` is interpreted. + +``` {#lst:filterbank_good .Rust float="" floatplacement="H" label="lst:filterbank_good" language="Rust" caption="\\it Example code that duplicates filter parametrically using a recursive function and closure."} +fn filterbank(n,filter){ + let next = filterbank(n-1,filter) + if (n>0){ + |x,freq| filter(x,freq+n*100) + + next(x,freq) + }else{ + |x,freq| 0 + } + } + let myfilter = filterbank(3,bandpass) + fn dsp(){ + myfilter(x,1000) + } +``` + +``` {#lst:bytecode_filterbank float="" floatplacement="H" label="lst:bytecode_filterbank" caption="\\it Compiled VM instructions filterbank example in Listing \\ref{lst:filterbank_good}"} +CONSTANTS[100,1,0,2] + fn inner_then(x,freq) + //upvalue:[local(4),local(3),local(2),local(1)] + GETUPVALUE 3 2 //load filter + MOVE 4 0 + MOVE 5 1 + GETUPVALUE 6 1 //load n + ADDD 5 5 6 + MOVECONST 6 0 + MULF 5 5 6 + CALLCLS 3 2 1 //call filter + GETUPVALUE 4 4 //load next + MOVE 5 0 + MOVE 6 1 + CALLCLS 4 2 1 //call next + ADDF 3 3 4 + RETURN 3 1 + + fn inner_else(x,freq) + MOVECONST 2 2 + RETURN 2 1 + + fn filterbank(n,filter) + MOVECONST 2 1 //load itself + MOVE 3 0 //load n + MOVECONST 4 1 //load 1 + SUBF 3 3 4 + MOVECONST 4 2 //load inner_then + CALLCLS 2 2 1 //recursive call + MOVE 3 0 + MOVECONST 4 2 //load 0 + SUBF 3 3 4 + JMPIFNEG 3 2 + MOVECONST 3 2 //load inner_then + CLOSURE 3 3 //load inner_lambda + JMP 2 + MOVECONST 3 3 //load inner_else + CLOSURE 3 3 + CLOSE 2 + RETURN 3 1 +``` + +# Discussion {#sec:discussion} + +As seen in the example of the filterbank, in , signal graph can be parametrically generated in an evaluation of global context, compared to that Faust uses a term-rewriting macro and Kronos uses a type-level computation as in the Table [1](#tab:comparison){reference-type="ref" reference="tab:comparison"}. + +The ability to describe both the generation of parametric signal processing and its content in a single semantics will make it easier for novice users to understand the mechanism of the language. Also, the single semantics may facilitate run-time interoperation with other general-purpose languages. + +On the other hand, there is the problem that the single semantics causes  to behave differently from the behavior expected in a normal lambda calculus. + +::: {#tab:comparison} + 1-3   Parametric Signal Graph Actual DSP + ---------------- ------------------------- ------------------ + 1-3 Faust Term Rewriting Macro BDA + 1-3 Kronos Type-level Computation Value Evaluation + 1-3 + Global Context + `dsp` Function + 1-3 + + : *Comparison of the way of signal graph generation and actual signal processing between Faust, Kronos and .* +::: + +## Different Behaviour Depending on the Location of Let Binding {#sec:letbinding} + +By having functions that have internal states which change over time in mimium, when higher-order functions are used, there is a counterintuitive behavior compared to general functional programming languages. + +Listing [\[lst:filterbank_bad\]](#lst:filterbank_bad){reference-type="ref" reference="lst:filterbank_bad"} is an example of the incorrect code slightly modified from the filterbank example in Listing [\[lst:filterbank_good\]](#lst:filterbank_good){reference-type="ref" reference="lst:filterbank_good"}. The difference between Listing [\[lst:filterbank_good\]](#lst:filterbank_good){reference-type="ref" reference="lst:filterbank_good"} and Listing [\[lst:filterbank_bad\]](#lst:filterbank_bad){reference-type="ref" reference="lst:filterbank_bad"} is that the recursive calls in the filterbank function are written directly, or once bound with `let` expression out of the inner function. Similarly, in the `dsp` function that will be called by the audio driver in mimium, the difference is whether the filterbank function is executed inside `dsp` or bound with `let` once in the global context. + +In the case of normal functional language, if all the functions used in a composition do not contain destructive assignments, the calculation process will not change even if the variable bound by `let` were manually replaced with its term (beta reduction), as in the conversion from Listing [\[lst:filterbank_good\]](#lst:filterbank_good){reference-type="ref" reference="lst:filterbank_good"} to Listing [\[lst:filterbank_bad\]](#lst:filterbank_bad){reference-type="ref" reference="lst:filterbank_bad"}. + +But in mimium, there are two major stages of evaluation, 0: the code is evaluated in the global environment (concretizing the signal processing graph) at first, and 1: the dsp function is repeatedly executed (actual signal processing) and the function may involve implicit internal state updates. Therefore, even though the code does not include destructive assignments, the recursive execution of the `filterbank` function is performed only once in Listing [\[lst:filterbank_good\]](#lst:filterbank_good){reference-type="ref" reference="lst:filterbank_good"} for the evaluation of the global environment, whereas in Listing [\[lst:filterbank_bad\]](#lst:filterbank_bad){reference-type="ref" reference="lst:filterbank_bad"}, every sample the dsp function is executed, the recursive function is executed and a closure is generated. Since the initialization of the internal state in the closure is performed at the time of closure allocation, in the example of Listing[\[lst:filterbank_bad\]](#lst:filterbank_bad){reference-type="ref" reference="lst:filterbank_bad"}, the internal state of the closure after the evaluation of `filterbank` is reset at each time step. + +``` {#lst:filterbank_bad .Rust float="" floatplacement="H" label="lst:filterbank_bad" language="Rust" caption="\\it Wrong example of the code that duplicate filter parametrically."} + + fn bandpass(x,freq){ + //... + } + fn filterbank(n,filter){ + if (n>0){ + |x,freq| filter(x,freq+n*100) + + filterbank(n-1,filter)(x,freq) + }else{ + |x,freq| 0 + } + } + fn dsp(){ //called by audio driver. + filterbank(3,bandpass) + } +``` + +``` {#lst:filterbank_multi .Rust float="" floatplacement="H" label="lst:filterbank_multi" language="Rust" caption="\\it Example of filterbank function using multi-stage computation in a future specification of mimium."} +fn filterbank(n,filter){ + .< if (n>0){ + |x,freq| filter(x,freq+n*100) + + ~filterbank(n-1,filter)(x,freq) + }else{ + |x,freq| 0 + } >. + } + fn dsp(){ + ~filterbank(3,bandpass) (x,1000) + } +``` + +This means that the major compiler optimization techniques such as the constant folding and the function inlining can not simply be appropriated for mimium. Those optimizations should be done after the evaluation of a global context and before evaluating `dsp` function. + +To solve this situation, introducing distinction whether the term should be used in global context evaluation (stage 0) and in the actual signal processing (stage 1) in type system. This can be realized with Multi-Stage Computation[@Taha1997]. Listing [\[lst:filterbank_multi\]](#lst:filterbank_multi){reference-type="ref" reference="lst:filterbank_multi"} is the example of `filterbank` code using BER MetaOCaml's syntaxes `..` which will generate evaluated program to be used in a next stage, and `~term` which embed terms evaluated at the previous stage[@kiselyov2014a]. + +`filterbank` function is evaluated in stage 0 while embedding itself by using `~`. This multi-stage computation code still has a same semantics in a generative signal graph generation and execution of the signal processing, in contrast to that Faust and Kronos. + +## A Possibility of the Foreign Stateful Function Call + +The data structure of closure in  is a combination of functions and internal states, as shown in Figure 3. The fact that +`filterbank` samples do not require any special handling of internal states also means that external signal processor (Unit Generator: UGen) such as oscillators and filters written in C or C++, for example, can be called from mimium in the same way as normal closure calls, and it is even possible to parametrically duplicate and combine external UGens. This is an advantage that is difficult to implement in Faust and other similar languages, but easy to implement on  paradigm. + +However currently, mimium is based on sample-by-sample processing and cannot handle buffer-by-buffer value passing. Since most native unit generators perform processing on a buffer-by-buffer basis, there are not many cases where external UGens are utilized in practice for now. However, in the , only $feed$ terms need to be processed sample-by-sample, so it is possible to distinguish functions that can only process one sample at a time from functions that can process concurrently at the type level. As the Multi-rate specification is being considered in Faust[@jouvelotDependentVectorTypes2011], it may be possible to read/write buffer between an external Unit Generator by having the compiler automatically determine the parts that can be processed as buffer-by-buffer. + +# Conclusion {#sec:conclusion} + +This paper proposed , an intermediate representation for the programming languages for music and signal processing with the virtual machine and instruction set to run it.  enables to describe generative signal graph and its contents in a single syntax and semantics. However, user have to be responsible to write codes that does not create escapable closures during the iterative execution of DSP, which will be difficult to understand for novice users. + +In this paper, the translation from  terms from VM instructions is explained by just showing examples of the code and its expected result of instructions as well as the semantics of VM is presented with pseudo-code of the behaviour. More formal semantics and translation process should be considered along with an introduction of the multi-stage computation. + +I hope that this research will lead to more general representations of music and sound on the digital computer and more connections between the theory of languages for music and more general programming language theory. + +# Acknowledgments + +This work was supported by JSPS KAKENHI (Grant No. +JP19K21615). Also great thanks for many anonymous reviewers. + +[^1]: Reason for this is that it is easy to implemented on `enum` data structure on Rust, a host language of the latest mimium compiler. Operands bitwidth and alignment may be changed in the future.