Programs

Misty has direct support for processes. Processes run independently and concurrently. Processes communicate by sending messages. Every process has its own memory space. Processes do not share memory. Every process runs a program. A program can use modules. Many programs can work productively together.

misty "misty" space misty_type space name more_statements linebreak "end" space name

misty_type "program" "module"

Every program has a process object called @at sign that contains a the private address of the process itself and other powerful capabilities. A process can pass process address objects (including an attenuated version of @) as parameters to functions or in messages to other processes. If a process can acquire the process address object of another process, then it can send messages to it. Messages may contain numbers, texts, records, arrays, logicals, blobs, and process address objects.

An attenuated @ object is produced when @ is on the right side of a set statement, or when @ is passed an argument to a function, or when @ is included in an array literal or record literal. An attenuated @ object contains the private address of the process.

Misty programs are organized into source files. There are two types of Misty source files:

• module
• program

A module is a chunk of independent program. These can be used to build reusable libraries. The body of the module contains a string of statements. The statements in the body may not include if or do.

The last statement in a module is a return statement, which usually returns a function or a record containing functions. That return value is bound to the name in a use statement. A program file does not end with a return statement.

When a process is started, the statements in the program file are executed. The statements should start the execution of the process, which usually involves the setting of a receiver so that the process can receive messages. Other sorts of initialization may take place as well.

The program can pull in code from the module library by use of the use statement. A module executes its body, as a function does, and returns a value that will be bound to the name in the use staement. Typically, it will return a constructor function, but it can also return a record of functions. The value returned by a module will be stone.

Modules can also contain use statements. It is possible that two or more use statements might reference the same module. Should this occur, the module will be executed once, and all of the use statements referencing that module will bind the same value. If two processes reference the same module, the module will be executed for each process. Processes do not share modules with way programs do.

Modules can not have cyclical dependences. Module a can not use module b if module b uses module a.

In this example, the example program imports the app_master_2000 module, and designates its handler function as the receiver of messages for the process.

misty program example

use app: "app_master_2000"
call @.receiver(app.handler)

end example

Processes are started with the @.new(program) method. A process that starts another process is called an overling. A process started by an overling is called an underling. A process can be a underling to one process and an overling to many others.

Communication between processes happens exclusively with messages.

Messages are usually transmitted over some sort of connection.

Process object

A process address object contains the information needed to communicate with a process. A process object can be transmitted to other processes, even on other machines.

A process address object is an immutable black box. It can be used in a send statement to send a message to the process associated with the process object. Process address objects can be sent to other processes, giving them the capability to also send messages to the process associated with the process address object.

None of the contents of the process object are visible or accessible.

Example:

process?(@)                       # true
process?(my_process)              # true
record?(my_process)               # false
stone?(my_process)                # true
my_process = my_process           # true
my_process = your_process         # false (probably)

The process? function

process?(value)

The process? function gives true if the value is an process address object.

Process lifecycle

A process is created by another process by @.new(program) which returns a new private address object. Over its existence, a process will receive messages, which may cause it to change its state and send messages.

When a process stops, it will no longer send or receive messages. Ultimately, there are five ways that a process stops:

Halt

A process can stop itself by calling @.halt(). It may do this as a result of being told to do so by its overling or another trusted process, or because it has fulfilled its purpose. Any messages sent in this final turn will be put into the outgoing queue.

Disrupt

If an explicit or implicit disrupt occurs that is not handled, then the process stops. Any messages sent in this final turn will not be put into the outgoing queue.

Stop

An overling process may stop a underling by calling @.stop(underling). If the underling process is in the middle of executing a turn when it is stopped, any messages sent in that final turn will not be put into the outgoing queue.

Coupling

If a process is coupled to a process that stops, then it also stops. A process can couple itself to another process by calling @.couple(process). Every process is automatically coupled to its overling.

Disaster

The system crashes, or an earthquake disables the data center, or there is a nuclear sneak attack, or a software bug. Surviving processes will probably not be immediately notified of the disaster.

Messages

Processes communicate using messages only.

Incoming messages are queued by the Misty system and delivered in arrival order. The exceptions are system level messages like the stop message, which, if valid, will cause a process to immediately stop, even if there are undelivered messages waiting for it in the queue.

Some messages can be used to reply to the original sender of the message.

receiver function

The receiver method is given a callback function that will receive the process's messages. The callback function will receive a single argument, the message object. The callback function will not be given reply messages.

@ Methods

The @ object is only available in misty program files. The @ object is not available in misty module files, although the attenuated process object and some of the @ methods can be passed in. The @ object may contain these methods: clock, connection, contact, couple, delay, garbage, greeter, halt, new, random, random_bits, receiver, stop.

clock method

@.clock(function)

The clock method takes a function argument that will eventually be called with the current time in number form. See time.

connection method

@.connection(callback, process, configuration)

The connection method takes a callback function, a process object, and a configuration record for getting information about the status of a connection to the process. The configuration record is used to request the sort of information that needs to be communicated. This can include latency, bandwidth, activity, congestion, cost, partitions. The callback will be given a record containing the requested information.

contact method

@.contact(callback, record)

The contact method sends a message to a greeter on another machine to obtain a process object.

The callback is a function with a process parameter and a reason parameter. If successful, process will be bound to a process object. If not successful, process will be null and reason may contain an explanation.

The record can contain:

• Coordinates
  • Network address
  • Port number
• Authentication
  • Public Key
  • Identifier
  • Shared secret
• Specification
  • Service name
  • Feature request
  • Process identifier

couple method

@.couple(process)

The couple method causes this process to stop when another process stops. The couple method returns null.

call @.couple(sponsor)

delay method

@.delay(function, seconds)

The delay method is used to schedule the invocation of a function at a later time. Any value returned from the delayed invocation will be ignored. There is no guarantee that the function will ever be invoked. The delayed invocation will not interrupt normal processing. The invocation will be delayed until the process is waiting for a message. The delay method returns null.

The delay function immediately returns a cancel function. Calling the cancel function will cancel the delayed execution of the function, if it is not too late.

The seconds parameter speicifies when the invocation will occur, no sooner than seconds seconds after now. The seconds parameter must be a non-negative number or null which behaves as 0.

call @.delay(continuation, 0.1) 

garbage method

@.garbage(function)

The garbage method registers a function that will receive a message informing the process that it has probably become unreachable and useless. The process should finish its work and then @.halt(). The garbage method returns null.

greeter method

@.greeter(function, port)

A greeter is a special process with a public address that performs introduction services. It listens on a specified port for contacts by external processes that need to acquire a process object. The function will receive the record containing the request. The record can have a reply sent through it. A greeter can respond by beginning a new process, or finding an existing process, or by forwarding the contact message to another process. This is how distributed Misty networks are bootstrapped. The greeter method returns null.

halt method

@.halt()

The halt method allows a process to stop itself. This usually happens when it has completed its work. Any messages sent in this final turn will be put into the outgoing queue. The halt method does not return.

new method

@.new(callback, program, configuration)

The new function creates a new process. It takes a program text that identifies the program file that the new process runs. The text locates a program in the program section of the misty module database.

The configuration record contains fields having the names of the @ methods. If the field's value is true, then the new process will have that method in its own @ object. So, if the configuration contains a contact field that is true, then the new process is allowed to contact greeters to obtain process objects. However, if the overling does not have access to the method itself, it can not make it available to the underling.

The configuation record can also contain constants and process address records that will be put into the new processes @ record.

The callback function will be passed the underling process object, or null if something went wrong.

The current process is the overling of the new process, and it will be notified when the new process stops. The new process is an underling of the current process.

Example:

call @.new(
    callback
    "example.mst"
    {
        contact: true
        couple: true
        greeter: false
        keys: fresh_key_pair
        new: true
        overling: @
        receiver: true
    }
)

random functions

@.random()

The random function returns a number between 0 and 1.

@.random_bits()

The random_bits function returns a signed whole number in the range -36028797018963968 thru 36028797018963967 that contains 56 random bits.

receiver method

@.receiver(function)

The receiver method registers a function that will receive all messages sent to the process except for delay events, reply messages (which are sent to the send callback), and greeter contact messages. The receiver method returns null.

stop method

@.stop(process)

The stop method stops an underling. The stop method returns null.

Process Data Structures

Process Address Object

A process object is used with the send statement. It contains a process's private address. A message may contain process address objects, which will give the recipient process the capability to send messages to those processes at the private addresses.

There are three ways that a process can obtain the process address object of another process:

• by being constructed by @.new() with addresses
• by creating a process with @.new()
• by receiving a message containing a process's private address object

Message object

A message object is obtained from the callback function that is registered with @.receiver. It acts like an ordinary record.

When a message is sent using the callback form, the message may be used once as a process's private address for transmitting the reply.

Turns

Computation takes place in a process in a fragment of time called a turn. A turn starts with the receiving of a message. A function (such as the function registered with @.receive, @.greeter, @.clock, or a delay callback function) will run to completion. Any outgoing messages will be held until the turn completes successfully, at which time they go into the outgoing queue and are sent.

A process will not receive another message until the turn ends. Each turn will process exactly one message.

If a machine has multiple computation units, then it is possible for multiple turns of multiple processes to be going on simultaneously. Turns can be timesliced. There are no concurrency issues because processes do not share memory. They communicate with other processes and the world only by message passing.

Failure

Fail to a known condition.

In distributed systems, we can not be certain of failure. Failure may be presumed.

Failure is always an option.

Guests

Logging