Mosketch embeds a streaming server allowing to send (and receive) joints values to (from) your local network. This is useful for example when you want to animate in Mosketch and see the results, in real-time, in another 3D package. It also allows to send Json commands to control Mosketch from another application. MosketchForMaya Python script demonstrates how to stream joints values (orientation and translation) from Mosketch to Maya (and vice versa). The script will work on any single-rooted 3D character. Please, read the user documentation first and send any bug reports and feature request to our helpdesk.
IMPORTANT NOTE: this document focuses on streaming from Mosketch to Maya using PyMEL. However, it should be possible to adapt this for any other application.
Overview
In this section, we explain the important steps to follow when writing a script interacting with Mosketch streaming server.
Connect to Mosketch: as a developer, you first need to connect to the machine running Mosketch streaming server using its IP and port 16094.
Listen for Hierarchy initialisation packet: your socket should listen for incoming packets. If the connection is successful, you will receive a first packet of type Hierarchy. At this point, you might want to create a table to hold joint’s data.
Send acknowledgement packet: you must send an acknowledgment message back. Mosketch will otherwise consider that this client didn't receive the data properly. Once Mosketch has received the acknowledgment message, it will start sending joint’s values.
Listen for JointsUuids initialisation packet: you will receive a second packet of type JointsUuids. This will be useful in the future when it will be possible to pilot Mosketch by sending Json commands
Send acknowledgement packet: <TO DOCUMENT WHEN DONE>
Listen for incoming joint values: Mosketch sends joint’s rotations in different frames (World, Parent, ParentInWorld). <ADD LINK TO SECTION>
Send joint values to Mosketch: you may want to update the character pose in Mosketch
1. Connect to Mosketch
You might connect to Mosketch using QTcpSocket or any other valid TCP socket service. Mosketch streaming server runs on port 16094. It uses JSON as the exchange format. and may send different types of packet depending on the sent information. Please read following sections for more details.
IMPORTANT NOTE: depending on your platform, you might have to manually allow traffic on this port. When the connection is successful, you are added as a new peer and the initialisation packet is sent.
2. Listen for Hierarchy initialisation packet
The first packet you will receive from Mosketch is an initialisation packet which is of the type called Hierarchy. Hierarchy packets contain the current lig of joints names present in the Mosketch's scene. They are of the following format:
{"Joints":["joint1","joint2","joint3","joint5","joint4"],"Type":"Hierarchy"}
Mosketch expects an acknowledgement.
For an example, please refer to Python function named _got_data. When receiving the hierarchy, you might want to save different joints information. For example, you might want to save the mapping between Mosketch Uuids and Maya's joints name.
3. Send acknowledgement packet
Before streaming data, Mosketch will wait for an acknowledgement to make sure the connection is well-established. This is important to prevent missing packets if we are in auto mode. Mosketch also accepts several peers so it needs to know which one is correctly initialized.
When ready, you can send an acknowledgement packet with the following format:
{"Type": "AckHierarchyInitialized"}
4. Listen for JointsUuids initialisation packet
When Mosketch receives the acknowledgment packet, it will send all joint’s universal unique IDs with a packet of type ‘JointsUuids’. It is sent right after it has received the hierarchy acknowledgment. The purpose is to send all Mosketch internal unique IDs for all joints in the scene. We encourage to save those UUIDs to be able to use Mosketch commands. In particular, nodes in Mosketch are identified by their UUID. To manipulate nodes, you must pass UUIDs as a parameter to the commands.
JointsUUIDs packets are of the following format:
{"Joints":[{"joint1":"A8B47957-EB01-47B5-8BF4-422690293816"},{"joint2":"9C9F9CD5-14CE-4A07-9949-9D9E50FEFF8A"},{"joint3":"1263F818-7B0F-4BF9-B2C2-F515BD933152"},{"joint4":"199E3DCF-FD69-4DA2-802B-4A6A1A6AF774"},{"joint5":"07712085-DD56-4BDA-ABE4-C6E0047FD800"}],"Type":"JointsUuids"}
5. Send acknowledgement packet
At the point, Mosketch expects an acknowledgement too.
When ready, you can send an acknowledgement packet with the following format:
{"Type": "JointsUuidsAck"}
6. Listen for incoming joint values
From now on, you will receive packets of type JointsStream containing joints' values. You can receive one joint only or all the joints at once with their rotations and / or translations depending on their degree of freedom. These packets have the following format:
{"Joints":[{"Anatom":7,"LR":[0,0,0,1],"LT":[0,0,0],"Name":"joint1"},{"Anatom":5,"LR":[0,0,0,1],"Name":"joint2"},{"Anatom":5,"LR":[0,0,0,1],"Name":"joint3"},{"Anatom":5,"LR":[0,0,0,1],"Name":"joint4"},{"Anatom":5,"LR":[0,0,0,1],"Name":"joint5"}],"Type":"JointsStream"}
It contains an array of objects containing joints information:
“Name”: name of the joint
“Anatom”: identifier for anatomical type for the joint. It can have 2 values for the moment, 5 for 3 DoFs joints (rotation only) and 7 for 6 DoFs joints (rotation and translation).
“R”: rotation in quaternion form.
“T”: translation in (x, y, z) form. It is sent only for joint with an anatomical type of 7.
When you are done processing this information, you must send an acknowledgment packet of the form:
{"Type": "JointsStreamAck"}
Mosketch will then continue sending joints values.
7. Send joint values to Mosketch
You might want to send data to Mosketch to set the current character pose. To do so, create and send a packet of type JointsStream. In MosketchForMaya Python script, refer to Python function _update_mosketch() for more information.
Joints rotation space
Mosketch™ is sending the joint’s rotations in parent space by default. If you’d like to change this setting, you can use the command setStreamingJointSpace to choose between “Parent”, “World” and “ParentInWorld” space. Please have look in MosketchForMaya Python script for more information.
Initial Joint Orientation
You might want to discard or keep the initial joints orientation in Mosketch. By default, they are kept in Mosketch. Please have look in MosketchForMaya Python script for more information.
Logs
To help debugging your script, any time an incorrectly formed packet is received by Mosketch, it is logged. As an example, if you send such a packet:
{"Type": "WrongTypeForExample"}
Here what you’ll get in our log file:
(2018-06-27T15:54:27) - Warning -> receiving wrong packet type 'WrongTypeForExample'.