Exploring creative coding with Swift and SwiftUI

Creative coding reframes programming as an artistic practice, enabling practitioners to engage with expressive visual forms beyond the purely functionalist problem-solving paradigm that predominates in software development. This practice has a rich historical lineage dating back to the early computational experiments, with pioneers like Vera Molnár, Frieder Nake, and Georg Nees establishing foundational methodologies for what would evolve into a vibrant artistic field thanks to the introduction of frameworks and languages such as Processing and p5.js.

In the conceptual and practical space opened by creative coding, errors, unpredictable results and serendipity gain new weight and play a fundamental role in the process of creating software. Drawing on Ira Greenberg's notion of “happy coding mistakes” in his book Processing: Creative Coding and Computational Art, creative coders embrace procedural indeterminacy to reveal unforeseen aesthetic configurations through computational exploration. Greenberg, who directs the Center of Creative Computation at Southern Methodist University, articulates how intersecting iterative processes, free experimentation and productive "errors", frequently yield "unexpected and interesting stuff" emerging from code - establishing a framework that privileges discovery over deterministic outcomes.

This finds expression in the practices of exemplary practitioners who deliberately integrate indeterminacy into their computational systems. Zach Lieberman engages with what he describes as a "conversation with systems," where the algorithmic output frequently produces surprising results that exceed authorial intent. In his daily sketching practice, Lieberman follows the principle of "prioritizing iteration over novelty," allowing emergent properties to manifest through repetitive processes with subtle variations. Austrian software artist LIA similarly approaches code as a dialogic medium, describing her process as a "conversation" between artist and machine where the translation between concept and formal structure generates unexpected configurations through iterative refinement.

Beyond aesthetics, creative code can also operate as a form of cultural critique. When integrated with critical studies and digital humanities frameworks, it becomes a methodological approach for interrogating technology's role in constituting identity, social structures, and semantic constructs - deconstructing underlying ideological formations, as demonstrated by scholars like Mark C. Marino in Critical Code Studies and a great number of computational artists such as Lauren Lee McCarthy, whose performances like "LAUREN" interrogate surveillance and algorithmic living by positioning herself as a human version of Amazon's Alexa and Tega Brain, whose ecological interventions examine the intersection of computation with environmental systems.

In this article, we will explore the complexity of practical techniques layered behind computational art from a more pragmatic point of view and demonstrate how Swift and Xcode can bridge theory and practice in the pursuit of unexpected beauty to create digital expressions.

Swift and Xcode as a creative studio

As explored in a previous article "Getting Started with Creative Coding Using Swift and SwiftUI", the landscape of creative computation has historically been principally defined by established frameworks and environments such as Processing, p5.js, and openFrameworks - platforms that have cultivated robust communities and extensive pedagogical resources dedicated to artistic expression through code. Despite their substantial technical capabilities, Swift and Xcode have occupied a relatively peripheral position within this ecosystem, remaining somewhat underutilized as creative mediums. In this article, we'll show how Swift and SwiftUI together offer much more than just another option for creative coders - they provide a powerful and sophisticated toolkit with special advantages for creating computational art.

One of the significant technical constraints limiting non-coders from jumping into computational art has historically been, beyond the inherent difficulty of programming, the delay barrier that existed between writing code and simultaneously visualizing the related result. This barrier began to be dismantled in 2001 by Processing and its concept of code sketches, an environment designed with the aim of making programming a means for creating visuals easily approachable for beginners by focusing on the immediacy of visualizing the creations on the screen. Later, platforms like p5.js kept making the almost real-time visualization of coding experiments experience appealing to artists and designers.

Nowadays, at the heart of Apple's ecosystem lies a complete creative coding studio that can be used for the same computational aesthetic purposes. The combination of Xcode and SwiftUI live previews provides an environment that is flexible and open to experimentation in a way that is similar to the quick visualization of code snippets that Processing and p5.js already offered to the creative community.

By shifting from a strictly functional mindset to a creative one, Swift and Xcode become complementary instruments that lower the barrier between artists and programming when it comes to computational aesthetics generation: think of the clarity of Swift's expressive syntax, the SwiftUI’s declarative paradigms, the strong type system, and their performance characteristics that create a unique environment that invites artists to think in shapes and gradients, image, texts; Xcode itself and its live previews accelerate a feedback-driven workflow facilitating rapid iteration and allowing artists - coders to see changes instantly as they modify the code; components like Canvas enables drawing and some other such as TimelineView continuous animation, giving our static compositions life through motion by surfacing generative patterns and algorithmic compositions in real time; Metal integration can further extends possibilities into GPU-powered generative art, making the iPad and Mac potent surfaces for interactive installations. And that’s not only that. There’s room to go much more in-depth if starting to think about how interaction can be implemented using frameworks such as Vision to detect any sort of interactivity in real-time and play around with its effect on the co-created visual outputs.

Layering techniques in the pursuit of unexpected beauty

There's no single entry point into the realm of creative coding experimentation with Swift.

Even though each manifestation of computational art has its own aesthetic concerns, technical approaches, and cultural lineage, these traditions resist rigid categorization.

In most cases, it is a layering of techniques, approaches, and procedures that lays the foundation for computational aesthetics. This strategic superimposition constitutes one of the cores of generative art practice - not simply applying single algorithms in isolation, but orchestrating multiple procedural systems that interact in complex ways. Through the deliberate layering of these techniques - strategically combining different algorithmic processes, such as geometric transformations, color manipulations, and controlled randomness - artists generate complex and unexpected results that exceed the sum of their individual components.

Most computational artists from different decades followed a similar approach. The pioneer Vera Molnár put at the heart of their approach the marriage of geometric primitives such as circles, squares, and lines with the layering of different manipulation operations on the elements composing their artistic sequence with the aim of creating differentiation by injecting controlled or uncontrolled noise. It is in this context that computational artists pursuit the beauty: the effects of this unpredictable results may vary accordingly to all the parameters the artist is involving with their experimentation, the way they intersect together and affect one another produce changes that may make the beauty emerge. As stated by Ira Greenberg, "[…] It is often through these types of overlapping iterative processes that really unexpected and interesting stuff starts emerging out of the code […]" (Processing: Creative Coding and Computational Art, p.182).

In the following sections, we will systematically deconstruct and examine these multifaceted compositional layers - from fundamental drawing operations to complex interactive systems - revealing how their strategic integration within Swift and SwiftUI enables the emergence of sophisticated generative aesthetics that would be unattainable through singular procedural approaches.

Drawing, sequence and animation

To start simple, a great entry point for creative coding with Swift can be using immediate drawing mode to mirror what early computer artists such as Frieder Nake, Georg Nees, or Manfred Mohr did during the late ‘60s and the ‘70s when they issued explicit commands to their devices to generate primitive figures alterations. Canvas, a SwiftUI view, comes in handy to support immediate mode drawing for 2D graphics. It provides control over paths, shapes, gradients, and blend modes that can be sent out as commands to be drawn on display. 

To start drawing using Canvas, it’s better to create a custom shape model where all the properties are stored: the number of points the path should consist of, the stroke width and its color, and the background color. The more properties defined, the more differentiation can be created. Playing around with these properties allows you to create variations of the same custom path. In the example above, four variations of the same custom shape have been made by changing the number of control points of a Path object.

Differentiation can also be added when arrays or sequences of shapes start being composed, altering the way they are perceived. What happens when playing around with the number of shapes appearing per row and per column? How do they layer together? What can the final result be?

Geometric transformations

Building upon basic drawing and sequencing, we can layer in another powerful approach: procedural geometric transformations. This additional layer in our creative coding process opens up new dimensions of visual complexity and unexpected results. Manipulating the shapes by applying basic transformations such as rotation, distortion, scaling, or translation, regulated by algorithms, played another crucial role in the creative procedural processes.

Pioneering computational artist Manfred Mohr was among the first to systematically explore algorithmic transformations of simple geometric structures. His "Cubic Limit" series (1973-1976) deconstructed and transformed the cube through rotations and dimensional projections, creating unexpectedly elegant compositions from rigorous mathematical operations. Similarly, computer art pioneer Vera Molnár's "Transformations" (1976) applied systematic rotational and scaling operations to simple square forms, allowing subtle variations in the transformation parameters to generate rhythmic visual music on the page.

Beyond the digital domain conceptual artist Sol LeWitt created works based on simple rule sets that generated complex visual outcomes. His wall drawings consist of instructions like "Lines in four directions, each direction equally spaced," which, when executed, create rich visual fields from minimal rules.

In SwiftUI, these operations are unlocked and applicable to the shapes using the CGAffineTransform methods like rotated(by:) , scaledBy(x:y:) and translatedBy(x:y:) or the related GraphicsContext methods such as rotate(by:) , scaleBy(x:y:) and translateBy(x:y:). This type of operation can be applied to crafting visually complex systems. Layering them together can create unexpected new shapes.

What makes these systems compelling is their unpredictability; despite being deterministic, the complexity that emerges from simple rules often surprises even the creator. This quality of discovery, where the artist sets up conditions but cannot fully predict the outcome, creates a unique relationship between the creative coder and their creation.

Contemporary creative coder Saskia Freeke exemplifies this approach in her daily creative coding practice (active since 2016), where she builds complex visual systems through layered geometric transformations. Her work often combines simple shapes with systematic transformations that create mesmerizing, rhythmic patterns that emerge through the interaction of multiple algorithmic layers.

Recursive algorithms that produce self-similar structures reminiscent of natural growth are an evolution of a similar approach and have been used in art during the last century, just like in 1904 Koch's Snowflake that shows how simple iterative geometry yields intricate forms. 

Casey Reas, co-creator of Processing, explores the balance between order and randomness in works like "Process 5" where simple rules are perturbed by random variations to create complex visual systems that evolve in unpredictable ways.

By taking advantage of the Canvas view performative features, SwiftUI can support this recursion: functions can call themselves, drawing nested shapes that grow into fractals or organic structures, echoing nature's own logic. This layering of recursive transformations allows us to build yet another dimension of complexity in our pursuit of computational beauty.

Color Alteration

Color theory and interpolation lie at the heart of generative aesthetics. Techniques like hue interpolation, alpha blending, and gradient mapping help transform geometry into rich, expressive visuals. It's another layer that can make things more interesting and surprising.

Josef Albers' groundbreaking work "Interaction of Color" (1963) demonstrated how our perception of color is always relational: the same color appears differently depending on its surroundings, and its context defines its appearance. His systematic explorations of color relationships laid the groundwork for computational approaches to color theory.

Contemporary artists like Tauba Auerbach create works that explore color space as a mathematical structure. Her "RGB Colorspace Atlas" presents the three-dimensional structure of the RGB color model as a physical book, making abstract color relationships tangible.

In the digital realm, generative artists like Tyler Hobbs create algorithmic color systems that produce harmonious palettes through mathematical relationships. They often incorporate natural color principles like the golden ratio to create visually pleasing combinations.

Argentinian artist Manolo Gamboa Naon (also known as Manoloide) has gained recognition for his distinctive approach to algorithmic color practice, where he brings color as the fourth element to generative art practice, which the majority usually ignore. His work features complex compositions that demonstrate how overlapping procedural techniques with a specific attention to the color dimension can produce unexpected visual outcomes through "beautiful errors".

The enduring artistic fascination with color finds a new life in the hands of today’s creative coders. What was once confined to canvas, pigment, and perception now unfolds through code and interaction. SwiftUI serves as a natural extension of this lineage, bridging the gap between theory and practice.

With features such as color blending, gradient, the Canvas view itself and the related GraphicsContext properties like blendMode or opacity, creative coders can explore color in a manner similar to artists have for centuries. They can layer, adjust, and experiment with hues, saturations, brightnesses, and opacities to create both visual harmony and emotional impact.

What happens when playing around with different colors for each part the path consists of - like its stroke and fill? How will the shapes affect each other when adjusting the hue, the brightness and the saturation based on algorithmic color systems? And what about the mode in which the colors blend together when shapes layer and overlap each other? How would their opacity react?

Interaction

Building on our layered foundation of drawing, geometric transformations, and color manipulation, we can add yet another dimension to our creative coding practice: interaction. By incorporating responsive elements, we transform our compositions from static expressions into dynamic conversations, inviting the audience to become active participants and co-creators in the artistic process.

Interactive art has a rich history that predates digital technology. However, computational interactivity emerged as a distinct approach in the late 1960s with pioneering works like Myron Krueger's "Videoplace" (1974), which used custom computer vision systems to create responsive environments. This early work established principles that continue to influence interactive media art today.

Daniel Rozin's interactive "mirror" series, such as his "Wooden Mirror" (1999), transforms everyday materials into responsive surfaces that reflect viewers' movements through purely mechanical or digital means, blurring the boundaries between computation and physical form.

Collective TeamLab's immersive environments like "Borderless" (2018-present) demonstrate how interactive computational art can transform entire spaces, creating responsive ecosystems where artwork flows between rooms and responds to human presence in real-time.

Interactivity on Apple devices can be enabled in many ways depending on what type of interaction needs to be detected - from the Gesture protocol to the Vision framework and more customizable experiences. This capability transforms static values into responsive experiences, inviting user/audience participation and creating a dialogue between creator, code, and viewer.

In SwiftUI, this layer of interactivity can be seamlessly integrated with our previous layers of drawing, transformation, and color alteration, creating multi-dimensional works that evolve based on touch, motion, or even biometric inputs like Depth Camera or heart rate data from Apple Watch. Interactivity is as deeply embedded in Apple devices as it is fundamental to computational art itself.

By incorporating this final layer of interactivity, we complete a comprehensive approach to creative coding that spans basic visual elements to complex, responsive systems—each layer building upon and enhancing the expressive potential of those beneath it.

Motion

Building upon our foundational layers of drawing, transformation, color, and interactivity, we can add yet another dimension through motion - bringing our static compositions to life through time-based sequence and transformation. Regardless of the forms used in their compositions, computational artists have introduced motion into static elements, creating dynamic visual experiences that evolve over time.

The time-based generation of sequences where images or shapes evolve frame by frame has played a fundamental role in computational art. It allows one to visualize not only motion but also the variation—differentiation—coming from the alteration. This process laid the groundwork for interactive animation systems and dynamic visual behaviors.

Their conceptual roots can also be traced to historical art movements. Kinetic artists like Alexander Calder also worked with motion, designing mobiles whose suspended elements move with air currents, generating endless variations within a defined system. These physical sculptures echo the logic of particle systems: constrained components moving in response to external forces.

Computer animation pioneer John Whitney Sr. was among the first to systematically explore algorithmic motion in the 1960s, creating mesmerizing visual symphonies where abstract forms moved in harmonic patterns. His seminal work "Matrix III" (1972) demonstrated how mathematical principles could generate hypnotic visual rhythms through precisely timed transformations.

In Swift, this idea finds a direct technical expression through the TimelineView, a view that facilitates periodic redrawing and makes it easier to build generative sketches that evolve over time. With clear instructions on when to update visuals, TimelineView supports the creation of generative sketches that evolve over time, reflecting the logic of time-based computation at the core of early algorithmic art.

Motion, especially when combined with interaction, transforms generative graphics into dynamic systems. Oscillations, spring dynamics, and easing curves bring fluidity and responsiveness to forms. These behaviors, grounded in mathematical principles, are easily implemented in Swift through built-in support for trigonometric functions, vectors, and matrices. Such tools allow the modeling of organic movement - waves, collisions, swarm-like behaviors, and more - by applying physical principles such as force, attraction, repulsion, and friction to determine how visual elements behave over time.

By adding this layer of motion to our creative coding palette, we transform static compositions into living systems that evolve, respond, and surprise - continuing our journey through the layered techniques that form the foundation of computational aesthetics.

Noise, randomness, emergent behaviors

The next layer to add to our creative coding “toolkit” consists of incorporating noise and controlled randomness to transform deterministic compositions into systems that surprise even their creators. These techniques introduce an element of the unexpected, creating works that balance precision with unpredictability.

The use of randomness has a rich tradition in computational art where it is widely used as a means to introduce surprise. Again, Vera Molnár was a pioneer in this domain. In pieces like "Interruptions" (2015), she combines ordered systems with controlled interruptions, creating visual compositions that feel balanced between determinism and chance—a balancing act that lies at the heart of effective generative art.

Before digital art existed, painters like Cy Twombly and François Morellet explored systematic approaches to composition with elements of chance. Kelly's "Colors for a Large Wall" (1951) arranged colored squares in a seemingly random yet carefully orchestrated grid, while Morellet's "Random Distribution of 40,000 Squares" (1963) used a random number table to determine the placement of elements.

Controlled randomness is often implemented through noise functions like Perlin or Simplex noise. Unlike pure randomness, these noise functions create smooth, continuous values that change gradually across space or time. Applied to a composition, each element's position, size, rotation, or color can be subtly influenced by controlled randomness, creating compositions that feel both ordered and organic and evolve continuously, where geometric forms seem to breathe, pulse, or flow in patterns that feel natural despite their mathematical origins.

Jared Tarbell, a pioneer in the Processing community, created influential works like "Substrate" which use controlled randomness to generate crystalline growth patterns that feel both organic and mathematically precise.

Contemporary artist Anders Hoff creates work that masterfully balances precision with randomness. His "Sand Spline" series uses noise to create variations in line placement, resulting in drawings that appear to have been made by a slightly imperfect human hand rather than a computer.

By incorporating this layer of controlled unpredictability into our creative coding practice, we open up possibilities for works that continue to surprise us long after we've written the code. In Swift, these techniques can be implemented through the built-in randomElement() method, the Random protocol, or custom noise functions, allowing us to create compositions that strike that delicate balance between order and surprise - adding yet another dimension to our growing repertoire of layered techniques.

Data as material

Building upon our layered creative coding approach, we can incorporate yet another dimension: using data as an artistic medium. This layer transforms information from abstract numbers into visual expressions, not just revealing patterns and relationships that might otherwise remain invisible, but actively driving and influencing the artwork itself.

Using data as a material for artistic expression has become increasingly prominent as our world becomes more data-driven. This practice emerged from information visualization but extends far beyond practical representation to use data as a dynamic creative force.

Contemporary data artists like Giorgia Lupi practice what she terms "data humanism" - approaches to data visualization that emphasizes the human stories and contexts behind abstract numbers. Her project "Dear Data" with Stefanie Posavec created hand-drawn visualizations of personal data, demonstrating how subjective and expressive data representation can be.

What makes data particularly powerful as a creative material is how it can actively shape and drive the artwork rather than just being represented. Lauren Lee McCarthy's interactive installation "Follower" uses location data to create a relationship between strangers, where one person secretly follows another for a day, transforming surveillance data into a provocative social experience.

Brian Eno and Peter Chilvers' "Bloom" app uses touch input data to generate both visual patterns and musical elements, creating a system in which data becomes the bridge between sound and image.

Refik Anadol's data sculptures like "Melting Memories" use EEG data collected from subjects recalling childhood memories to drive complex visual simulations, creating flowing, dynamic projections that visualize the invisible processes of memory formation.

Tiziana Alocci, an Italian information designer and creative technologist, creates powerful data-driven artwork that transforms music and cultural data into immersive visual experiences. Her project "David Gilmour Live at Pompeii" visualizes the musical structure and emotional impact of a live performance, while "Tokyo Love Story" transforms dating data from Tokyo into abstract visual compositions that reveal cultural patterns of romance and connection. Her work exemplifies how data can be both visualized and used to drive aesthetic decisions, bridging analytical and emotional approaches to information.

Swift's strong typing and data handling capabilities make it well-suited for transforming raw data into dynamic visual form. Using SwiftUI's Canvas view, real-time data streams from sensors, APIs, or user interactions can be mapped and translated into visual attributes that evolve over time.

For sound-reactive visuals specifically, Swift provides access to audio input through frameworks like AVFoundation and AudioKit, which can analyze audio in real-time, extracting features like amplitude, frequency, and spectral characteristics. These audio parameters can then drive visual elements - making shapes pulse with the beat, colors shift with frequency, or particle systems respond to vocal input.

The challenge lies in finding meaningful and aesthetically compelling mappings between data and visual elements—transformations that not only reveal hidden patterns but also create dynamic visual systems that respond and evolve with the data source.

By adding this data-driven layer to our creative coding practice, we transform abstract information into tangible, responsive experiences that can inform, inspire, and provoke - continuing our exploration of how layered techniques enable rich computational expressions.

Conclusion

Creative coding is not about mastering tools - it’s about layering intention, logic, and play. With Swift and SwiftUI, the artist-programmer has a whole ecosystem to explore this space. Whether drawing geometric sequences, transforming forms recursively, injecting noise, animating with physics, or weaving interactivity and data into a canvas - each layer enriches the visual language. Once adopted a creative mindset, every Apple framework becomes a new brushstroke in your studio, think of what could be achieved by using SpriteKit, AVFoundation, ARKit, Metal, and beyond.

The pursuit of unexpected beauty often lies not in precision, but in letting go: embracing randomness, allowing the interaction to shape outcomes, and using parametrization to create infinite variations from a single idea - exposing variables for angle, scale, color and timing, it's an invitation to variation and iteration that turns one sketch into an infinite constellation of possibilities.

Controlled or uncontrolled, randomness breathes life into rigid systems. Interactions - gesture, voice, movement - add unpredictability, turning compositions into evolving conversations. Layering these techniques creates complexity from simplicity, enabling artworks that feel organic, responsive, and alive. Creative coding, then, becomes an exploration of possibility: a way to think critically, experiment freely, and discover beauty not by controlling every outcome, but by designing systems that invite the unexpected to emerge.