Introduction to GUI

Basic elements: Windows, views and containers

The most fundamental element of the GUI is the Window. It occupies a rectangular space on screen within which other GUI elements are displayed. It usually has a bar that displays the window's title and allows for moving it, resizing it and closing it with the controls it displays or through mouse and keyboard interaction. Some of these aspects may be controlled within SuperCollider GUI code, though it is largely platform-dependent how precisely interaction with a window happens and is visually indicated.

The GUI elements contained within a Window are called views. They all inherit from the basic View class. The view occupies a rectangular space of the window within which it draws itself to display some data or to indicate a mode of interaction between the user and the program. Views receive keyboard and mouse events generated by the user and respond to them by controlling the behavior of the program. They also display information about the state of the program and the data on which it operates.

There are also special types of views that can contain other views and are thus called containers, for example the CompositeView. They allow for structuring GUI in a hierarchical way. A container view is called a parent of the views it contains, and they are called its children. Hierarchical organization allows to easily change aspects of all the views within a container: if the parent view is hidden, so are all the children; if the parent view is moved, so are they. Children are positioned with coordinates relative to their parent.

The following example shows a window containing a Button, a Slider and a group of StaticText views contained in a CompositeView. When the button is clicked the visibility of the CompositeView is toggled, while interacting with the Slider will move the CompositeView (and consequently all its contents) in horizontal direction.w = Window.new("GUI Introduction", Rect(200,200,255,100)); b = Button.new(w,Rect(10,0,80,30)).states_([["Hide"],["Show"]]); l = Slider.new(w,Rect(95,0,150,30)); c = CompositeView.new(w,Rect(20,35,100,60)); StaticText.new(c,Rect(0,0,80,30)).string_("Hello"); StaticText.new(c,Rect(20,30,80,30)).string_("World!"); b.action = { c.visible = b.value.asBoolean.not }; l.action = { c.bounds = Rect( l.value * 150 + 20, 35, 100, 100 ) }; w.front;

Automatic positioning and resizing of views

As a handy alternative to specifying all the dimensions and positions of views explicitly in code, SuperCollider allows for automatic positioning and resizing of views in relation to each other and in relation to window size - at the view creation and dynamically, when window is resized. There is several mechanisms for this purpose.

View's resize options

Views can automatically resize or move when their parent is resized, in one of the nine different ways that define how each of the view's edges will move along with the parent's edges. For documentation see the view's resize method and Resize behaviour document.w = Window.new("GUI Introduction", Rect(200,200,200,200)); TextField.new(w,Rect(0,0,200,30)).resize_(2); Slider.new(w,Rect(0,30,30,170)).resize_(4); TextView.new(w,Rect(30,30,170,170)).resize_(5); w.front;

Decorators

Decorators are objects that can be assigned to container views to carry the task of positioning the container's child views (currently there exists only one: FlowLayout). After a decorator is assigned to a container, the views created as its children will automatically be positioned in a specific pattern. See documentation of FlowLayout for details.w = Window.new("GUI Introduction", Rect(200,200,320,320)).front; // notice that FlowLayout refers to w.view, which is the container view // automatically created with the window and occupying its entire space w.view.decorator = FlowLayout(w.view.bounds); 14.do{ Slider(w, 150@20) };

Layouts

Layout classes make part of a complex system to manage both position and size of views. Using layouts, only relations of views within a pattern of organization need to be specified and their exact positions as well as sizes will automatically be deduced based on their type (the content they display and the type of interaction they offer) and in accord with principles of good GUI usability. Layouts also position and resize views dynamically, whenever their parent is resized or their contents change.

See the Layout Management guide for detailed explanation.

Customizing appearance

Views offer various ways to customize their appearance. This ranges from decorative changing of colors they use to draw themselves to controlling how they display various kinds of data.

Colors

Colors are represented in GUI code by the Color class.

A typical color that can be customized is background color - a color of choice can be applied to whatever is considered to be the background of a particular view. Views that display some text will typically also allow customizing its color as well.

Custom colors may be associated with different changing states of views or data they display, for example: Button allows to associate background and text colors with each one of its states, and will thus switch colors together with state when clicked; ListView allows to set a different background color for each of its items, as well as special background and text colors applied only to the item currently selected.

Whenever you execute the following example, random colors will be applied to different aspects of the views:( w = Window("GUI Introduction").background_(Color.rand).front; b = Button(w, Rect(10,10,100,30)).states_([ ["One",Color.rand,Color.rand], ["Two",Color.rand,Color.rand], ["Three",Color.rand,Color.rand] ]); l = ListView.new(w, Rect(10,50,200,100)) .items_(["One","Two","Three"]) .colors_([Color.rand,Color.rand,Color.rand]) .hiliteColor_(Color.blue) .selectedStringColor_(Color.white); r = Slider(w, Rect(10, 160, 200, 20)) .knobColor_(Color.rand) .background_(Color.rand); )

Palette

In Qt GUI, the complete set of colors used to draw the views is represented by a palette (see the QPalette class). Using a palette, you can define (most of) the appearance of the whole GUI in one go.

In the following example, clicking on the button will switch between two palettes. Note however, that the color assigned to the first Button state will beat the red color defined in the palette, and that colors of individual ListView items are not controlled by the palette.( x = QPalette.auto(Color.red(0.8), Color.red(0.5)); y = QPalette.auto(Color.cyan(1.4), Color.cyan(1.8)); p = QtGUI.palette; QtGUI.palette = x; w = Window.new("GUI Introduction").front; w.onClose = {QtGUI.palette = p}; Button.new(w, Rect(10,10,100,30)).states_([ ["Red", Color.black, Color.grey(0.7)], ["Cyan"] ]).action_({ |b| QtGUI.palette = if(b.value == 0){x}{y} }); ListView.new(w, Rect(10,50,200,100)) .items_(["One","Two","Three"]) .colors_([Color.grey(0.4),Color.grey(0.5),Color.grey(0.6)]); Slider(w, Rect(10, 160, 200, 20)); RangeSlider(w, Rect(10, 190, 200, 20)); )

Fonts

Views that display some text will typically allow you to specify a custom font for it. Fonts are represented by the Font class, which can also be queried for the default font used in general, as well as the default font specifically for the "serif", "sans-serif" and "monospace" font types. It can also be queried for all available fonts on the system.( w = Window.new("GUI Introduction",Rect(200,200,200,70)).front; a = [Font.defaultMonoFace, Font.defaultSansFace, Font.defaultSerifFace]; b = Button.new(w,Rect(10,10,180,50)) .states_([["Monospace"],["Sans serif"],["Serif"]]) .font_(a[0]) .action_({|b| b.font = a[b.value]}); )

Other visual properties

Complex views may have many other ways to customize how they display the same data. MultiSliderView and EnvelopeView are good examples.

Actions and hooks: Make that button do something!

Views and windows can be assigned actions that they will perform whenever a specific event occurs as a result of user's interaction. Technically, an action can be any Object, and when the relevant event occurs, it's value method will be called. For example, it is useful to assign a Function as an action, which allows one to define an arbitrary chunk of code to be performed in response to a GUI event.

Objects can also be given to views and windows to evaluate on events that are not a direct result of user's interaction, but convey useful information about the view's operation and the state it moved in. In this case they are often differentiated from actions and called hooks.

Here, we will give an overview of different kinds of actions and hooks. See View: Actions in general and following sections for precise explanation of how to assign and make use of them.

Default actions

Views can typically be assigned a default action with their action setter method, which will be performed when the view's primary mode of interaction is invoked. The default action for a Button for example occurs when it is clicked, for a Slider when its handle is moved.

In the following example, pressing the button will open an exact same window but at different position.~makeWindow = { var w; w = Window.new("Evader",Rect(500.rand + 100, 500.rand + 100, 200,50)).front; Button.new(w,Rect(10,10,180,30)).states_([["Evade"]]).action_(~makeWindow); }; ~makeWindow.value;

Keyboard and mouse actions

All the views can be assigned actions to specific mouse and keyboard events, no matter what other effects those events might have on the view or what other specialized actions or hooks the view might trigger on these events.

You can assign actions to mouse events generated when the mouse pointer enters the space of a view, when it moves over them, and when a mouse button is pressed or released.

See View: Mouse actions for details.

In the following example the StaticText will report whether the Button is pressed or released.w = Window.new(bounds:Rect(200,200,200,50)).front; b = Button.new(w,Rect(10,10,80,30)).states_([["Off"],["On"]]); t = StaticText(w,Rect(100,10,90,30)).string_("Button released"); b.mouseDownAction = { t.string = "Button pressed" }; b.mouseUpAction = { t.string = "Button released" };

You can assign actions to keyboard events generated whenever a key is pressed or released while the view has keyboard focus. Keyboard focus is a state of a view in which it has exclusive priority to respond to keyboard events. A view that has keyboard focus typically in a way visually indicates so. On most platforms, pressing the Tab key will switch the keyboard focus between views in the active window and clicking on a view will give it focus.

See View: Key actions for details.

Typing text into any of the TextFields in the following example will change the color of the rectangle bellow, for each TextField a different color.w = Window.new(bounds:Rect(200,200,200,100)).front; x = TextField(w,Rect(10,10,80,30)); y = TextField(w,Rect(110,10,80,30)); t = StaticText(w,Rect(10,40,180,50)); ~reset = {t.background = Color.red}; x.keyDownAction = {t.background = Color.green}; x.keyUpAction = ~reset; y.keyDownAction = {t.background = Color.blue}; y.keyUpAction = ~reset; ~reset.value;

If a key or mouse event is not handled by the view on which it occurs, it may propagate to the parent view, and trigger the parent's action. See View: Key and mouse event processing for detailed explanation.

Drag and drop actions

When a mouse button is pressed on a view together with Cmd(macOS) or Ctrl(Other OS) key and the mouse pointer is moved while holding the button, a drag-and-drop operation is initiated - in case the view supports it. Most views have a default object that they export when a drag is attempted. For a Slider it is its value, for a List it is the numeric index of the currently selected item, etc. It is said that the exported object is being dragged. When the dragging gesture ends on another view by releasing the mouse button on top of it, it is said that the dragged object was dropped on another view. A view may respond to various objects dropped on it in different ways.

It is possible to customize what object a view exports when dragged from and how a view reacts to objects dropped by assigning custom drag and drop actions.

See View: Drag and drop for details.( w = Window.new.front; a = Button(w, Rect(10, 10, 200, 20)).states_([["Hi There!"]]); a.beginDragAction = { a.dragLabel ="I'm dragging: \""++ a.states[0][0]++"\""; a.states[0][0] }; DragSink(w,Rect(10,40,200,20)).align_(\center).string="Cmd-drag from Button to here"; )

Other specialized actions

Some views can be assigned actions on other events specific to their mode of interaction with the user which you are invited to discover by consulting their documentation.

Hooks

Hooks are various events that signify important changes of state of GUI elements. Technically they are used the same way as actions, but are distinguished from them to denote events that are not a direct result of the user's interaction. Methods of GUI classes used to assign hooks are usually prefixed with "on". (You will also find this naming pattern in methods of other SuperCollider classes, that have hooks in the same sense).

For example, one hook that every view as well as Window has is onClose, which is triggered when the window is closed or the view is removed. Other hooks for example exist for the case when a Window becomes or ceases to be the active one.

Custom views

The UserView is a view that displays and does nothing on itself, but allows you to define how it will be drawn, and for which you can define the entire behavior using mouse, key, and drag and drop actions. For documentation on all of these aspects, see UserView, View, and Pen. The explanation below, however, will demonstrate the basic techniques for designing a custom view.

You will be using the Pen class to draw the view. Pen is a powerful class that allows you to algorithmically draw using simple visual primitives like lines, arcs, curves, rectangles, ellipses, etc. and fill the shapes with colors and gradients.

Creating a custom view involves the following steps:

1. create a User View
2. define a draw function
3. define the default action
4. define mouse actions
5. define key actions
6. define drag and drop actions

You can omit steps which you don't need.( var value = 0.5; w = Window.new.front; // (1) create a UserView v = UserView(w,Rect(50,50,200,20)); // (2) define a drawing function using Pen v.drawFunc = { // Draw the fill Pen.fillColor = Color.grey; Pen.addRect(Rect(0,0, v.bounds.width*value,v.bounds.height)); Pen.fill; // Draw the triangle Pen.fillColor = Color.red; Pen.moveTo(((v.bounds.width*value)-5) @ v.bounds.height); Pen.lineTo(((v.bounds.width*value)+5) @ v.bounds.height); Pen.lineTo(((v.bounds.width*value)) @ (v.bounds.height/2)); Pen.lineTo(((v.bounds.width*value)-5) @ v.bounds.height); Pen.fill; // Draw the frame Pen.strokeColor = Color.black; Pen.addRect(Rect(0,0, v.bounds.width,v.bounds.height)); Pen.stroke; }; // (3) set the default action v.action = {value.postln; v.refresh}; // (4) define mouse actions v.mouseDownAction = { arg view, x = 0.5,y, m; //m.postln; ([256, 0].includes(m)).if{ // restrict to no modifier value = (x).linlin(0,v.bounds.width,0,1); v.doAction}; }; v.mouseMoveAction = v.mouseDownAction; // (5) (optional) define key actions v.keyDownAction = { arg view, char, modifiers, unicode,keycode; if (unicode == 16rF700, { value = (value+0.1).clip(0,1) }); if (unicode == 16rF703, { value = (value+0.1).clip(0,1) }); if (unicode == 16rF701, { value = (value-0.1).clip(0,1) }); if (unicode == 16rF702, { value = (value-0.1).clip(0,1) }); v.doAction; }; // (6) (optional) define drag and drop behavior v.beginDragAction = {value}; // what to drag v.canReceiveDragHandler = {View.currentDrag.isNumber}; // what to receive v.receiveDragHandler = {value = View.currentDrag; v.doAction }; // what to do on receiving // just for testing drag and drop Slider(w,Rect(50,100,200,20)); StaticText(w,Rect(50,150,350,50)).string_("To Test Drag and Drop,\nHold down Cmd (Ctl) Key"); )

Caution: GUI and timing

If you attempt to interact with a GUI object in the contexts listed above, an error will be thrown.

Therefore, if you want to use Functions, Routines, Tasks and other similar objects to schedule code that interacts with GUI elements, you must do so using the AppClock, since code scheduled on the AppClock is performed in the main application context. You can of course also reschedule GUI code to the AppClock from within code performed in other contexts, and the 'defer' mechanism is a convenient shorthand for this.

An example of scheduling GUI code on the AppClock:w=Window.new.front; Routine{ 20.do{ w.bounds=Rect(200.rand, 200+200.rand, 300,300); 0.1.wait; }; w.close; }.play(AppClock)

The same thing using the SystemClock in combination with the defer mechanism:w=Window.new.front; Routine{ 20.do{ {w.bounds=Rect(200.rand, 200+200.rand, 300,300) }.defer; // you must defer this 0.1.wait; }; {w.close}.defer; // you must defer this }.play(SystemClock)

As mentioned above, using the GUI system is also not allowed in code performed directly in response to OSC messages (this includes functions given to all kinds of OSC responder classes). The same solutions as above apply.

Another example for addressing this issue can be found in the FAQ Language (client) Issues