A PixiAnimate Aquarium

Andrew Rapo
November 11, 2016

Note: This post elaborates on a previous post which provides an introduction to using the PixiAnimate Extension.

Timeline Animation

This post illustrates the way Adobe Animate and the PixiAnimate Extension can be used to create programmatically-controlled, game-like timeline animations for Jibo. For example, consider a hypothetical Jibo Aquarium - featuring Jibo-inspired fish created by Lead Designer Fardad Faridi. 
Aquarium_1.png

Video Below:



The Main Timeline

The Adobe Animate FLA source file for this example (actually an xfl document) has a main timeline, which defines the visual hierarchy of the aquarium, including the blue background (bg), the food, fish and plant layers and the foreground (fg), which in this case is a black rectangle with a circular window. When rendered, these layers are drawn back to front and help create the illusion of depth. Each Symbol (art element) on the main timeline has a unique instance name. These fish are named: fish_orange, fish_blue, and fish_purple and these instance names can be used by the code to reference and manipulate these Symbols.
Aquarium_Animate_MainTimeline.png

The Fish MovieClip Timeline

Symbol that contains the animation timeline for specific objects, like a Fish, are also referred to as a MovieClips. MovieClips can be manually placed on the Main Timeline in Adobe Animate - as in the example above - or they can be created programmatically via JavaScript code. Both techniques are used in this example.

In the same way that the Main Timeline determines the visual layering of the bg, fish, fg, etc., the Fish Timeline determines the visual layering of the Fish’s tail, fins, body, and eyes. The Fish in this example is made up of a number of Symbols that can be animated independently on the timeline.

Video Below:

Timeline Labels

When the Aquarium is running, JavaScript code will control the Fish animation using Timeline Labels. Timeline Labels serve as markers for important points in the animation. The timeline for the Fish in this example has labels that mark different modes of swimming: idle, slow and fast. The idle animation starts with the label, idle, and ends with the label idle_loop. The same is true for the slow and fast animations (the slow_loop and fast_loop labels are not expanded so they cannot be seen in the screenshot).

Publishing the Aquarium Animation

When using the custom PixiAnimate document type for Adobe Animate, publishing the document will generate JavaScript code that can be used to render the animation on Jibo (or in any Web environment that supports WebGL and PixiJS). In addition to the code, publishing also generates a spritesheet containing the animation’s bitmap assets.
Aquarium_Animate_PublishSettings2.png

Publish Settings

The publish settings dialog in Adobe Animate shows that the Aquarium animation will be exported to the JavaScript file, timelines/aquarium.js, and the spritesheet will be generated in timelines/images.

aquarium_atlas_1.png

Using the Aquarium Animation in a Skill
Once the animation is exported it can be incorporated into a Jibo Skill. Skills are authored using GitHub’s Atom editor with the Jibo SDK package installed. The Jibo SDK enables a number of custom file types including Flows (.flow files). Flows provide a visual way to manage the logical flow of a skill. They are especially useful for authoring dialog interactions (more about that in a future post). The main.flow file in this example is used to set up the Aquarium and run its main loop.

Note: Flows are compiled into JavaScript as part of the skill’s build process.
Aquarium_SDK_Layout.png

The code for the skill is organized in the source tree of the project and includes the index.ts which defines the main Aquarium Class, and main.flow
Aquarium_SourceTree.png

The Aquarium Skill is launched by creating an instance of the Aquarium Class and invoking its open() method. The open() method instantiates and runs the mainFlow (main.flow). Notice that jibo.flow.run() is configured using an options object. The options.blackboard property provides a way for the Aquarium Skill to share data with the main.flow. In this case, options.blackboard.skill points to the running instance of the Aquarium Skill. Code in main.flow can use this reference to invoke the Aquarium Skill’s methods.

Note: The best practice is to organize code in class files (i.e index.ts) and reference this code in Flows. 
Aquarium_open.png

addQuariumView()

The Aquarium Class (index.ts) defines an addAquariumView() method which instantiates a new View using jibo.face.views.addView. Views are part of the SDK’s GUI Kit and take care of loading and rendering animations.
Aquarium_addAquariumView.png

The addView method references a JSON view definition file that specifies the location of the aquarium.js file that was generated by Adobe Animate.
Aquarium_AquariumViewJSON.png

Calling addAquariumView() from within main.flow

The code in the Aquarium’s main.flow is organized using ‘Eval’ blocks which can contain arbitrary JavaScript. The first of these is named AddView. As noted above, the AddView block makes use of the blackboard property to invoke the addAquariumView() method of the Aquarium Skill.
Aquarium_Flow_AddView.png

Taking Control of Fish

The  GetFish Eval block invokes the Aquarium Skill’s getAquariumItemWithInstanceName() method which finds the Symbol with a given instance name (fish_orange) on the Main Timeline of the aquarium.js animation. A reference to fish_orange is stored in notepad.orangeFish. Like blackboard, notepad is an object that can be used to share data. Every Flow document has its own notepad. Properties added to notepad can be accessed anywhere in a given Flow - but not outside the scope of that Flow.
Aquarium_Flow_GetFish.png

Aquarium_getAquariumItemWithInstanceName.png

Using the reference to fish_orange in notepad.orangeFish, the AddExistingFish Eval block invokes the Aquarium Skill’s addExistingFish() method. This allows the AquariumManager to take control of a fish that already exists on the Main Timeline of the aquarium.js animation.
Aquarium_Flow_AddExistingFish.png

Aquarium_AddFish.png

Adding Fish Dynamically

Then the AddDynamicFish Eval block invokes the Aquarium Skill’s addDynamicFish() method to create a new instance of a fish from the library of of the aquarium.js animation. The library contains the definitions of all the Symbols defined in the Aquarium FLA (xfl) file. Each Symbol can be referenced by the name given to it in the library.

Aquarium_Animate_Library.png

Aquarium_Flow_AddDynamicFish.png

The Aquarium’s Main Loop

Finally, the MoveFish Eval block invokes the Aquarium Skill’s updateAquarium() method which animates the fish. Then the Loop Eval block re-invokes the MoveFish block (infinitely).

Aquarium_Flow_MoveFish.png
Aquarium_UpdateAquarium.png

More to Come

This relatively simple example provides an introduction to programmatically controlled animation using PixiAnimate as well as an introduction to authoring techniques using Jibo’s custom Flow documents. As noted above, the full power of Flows becomes apparent when they are used to author complex interactions like dialog. These concepts will be discussed in greater detail in future posts.

For those who are interested in looking a little deeper, more of the source code for this example is provided below.

Appendix

index.ts (the Aquarium Class)

/// <reference path="../typings/index.d.ts" />
import {BeSkill} from '@be/be-framework';
import jibo = require('jibo');
import AquariumManager from './aquarium/AquariumManager';


let mainFlow:any = require('./flows/main');


class Aquarium extends BeSkill {


    public flow:any;
    public blackboard:any;
    public aquariumView:any;
    public aquariumClip:any;
    public aquariumTimeline:any;
    public aquariumLibrary:any;
    public aquariumMovieClip:any;
    public aquariumPreviousTime:number;
    public aquariumTotalTime:number;
    public currentAnimationLabel:string;


    constructor(assetPack?:string) {
        super(assetPack);
        this.flow = null;
    }


    preload(done:(err?:any)=>void):void {
        done();
    }


    open(result?:any):void {
        this.blackboard = {
            skill: this
        };
        const options = { assetPack: this.assetPack, blackboard: this.blackboard };


        if (mainFlow) {
          this.flow = jibo.flow.run(mainFlow, options, () => {
            this.exit();
          });
        }
    }


    updateAquarium():void {
        let currentTime:number = new Date().getTime();
        let frameTime:number = currentTime - this.aquariumPreviousTime;
        this.aquariumPreviousTime = currentTime;
        this.aquariumTotalTime += frameTime;
        AquariumManager.update(frameTime, this.aquariumTotalTime);
    }


    addAquariumView(done):void {
        this.log.info(`addAquariumView:`);
        jibo.face.views.addView('resources/views/aquarium.json', (view) => {
            this.aquariumView = view;
            this.aquariumClip = (<any>view.getComponentById('aquariumClip'));
            this.aquariumMovieClip = this.aquariumClip.movieClip;
            this.aquariumTimeline = this.aquariumClip.timeline;
            this.aquariumLibrary = this.aquariumTimeline.library.library;


            this.aquariumPreviousTime = 0;
            this.aquariumTotalTime = 0;


            AquariumManager.init(this.aquariumView.stage);
            done();
        });
    }


    addExistingFish(id, mc):void {
        AquariumManager.addExistingFish(id, mc);
    }


    addDynamicFish(id, className):void {
        let fishClass:any = this.aquariumLibrary[className];
        AquariumManager.addFish(id, fishClass);
    }


    getSubClipWithInstanceName(movieClip:any, instanceName:string):any {
        let result:any = null;


        if (movieClip) {
            result = movieClip[instanceName];
        }
        return result;
    }


    getAquariumItemWithInstanceName(instanceName:string):any {
        let result:any = null;


        if (this.aquariumClip) {
            result = this.aquariumMovieClip[instanceName];
        }
        return result;
    }


    removeAquariumView(done):void {
        this.log.info('removeView');


        if (this.currentAnimationLabel) {
            let outLabel:string = this.currentAnimationLabel + '_out';
            PIXI.animate.Animator.play(this.aquariumMovieClip, outLabel, () => {
                this.log.info('played: ' + outLabel);
                jibo.face.views.removeView();
                this.aquariumView = null;
                done();
            });
        } else {
            jibo.face.views.removeView();
            this.aquariumView = null;
            done();
        }


    }


    close(done:() => void):void {
        if (this.flow) {
            this.flow.destroy();
            this.flow = null;
        }
        this.blackboard = null;
        this.aquariumView = null;
        this.aquariumClip = null;
        this.aquariumTimeline = null;
        this.aquariumLibrary = null;
        this.aquariumMovieClip = null;
        done();
    }
}


module.exports = Aquarium;

AquariumManager.ts

import Fish from './Fish';
import FishFood from './FishFood';


export default class AquariumManager {


    static stage:any;
    static aquariumStage:any;
    static backgroundLayer:any;
    static fishLayer:any;
    static foregroundLayer:any;
    static fish:any[];
    static food:any[];
    static startTime:number;
    static previousTime:number;
    static initialized:boolean;


    static init(stage) {
        console.log(`AquariumManager: init`);
        console.log(stage);
        AquariumManager.stage = stage;
        AquariumManager.aquariumStage = stage.children[0].children[0];


        AquariumManager.backgroundLayer = new PIXI.Container();
        AquariumManager.fishLayer = new PIXI.Container();
        AquariumManager.foregroundLayer = new PIXI.Container();
        AquariumManager.aquariumStage.addChild(AquariumManager.fishLayer);
        AquariumManager.aquariumStage.addChild(AquariumManager.foregroundLayer);
        AquariumManager.foregroundLayer.addChild(AquariumManager.aquariumStage['fg']);


        AquariumManager.fish = [];
        AquariumManager.food = [];
        AquariumManager.startTime = 0;
        AquariumManager.previousTime = 0;
        AquariumManager.initialized = true;
    }


    static addFish(id:string, fishClass:any):Fish {


        let newFishMC:any = new fishClass();
        newFishMC.x = 640;
        newFishMC.y = 320;
        return AquariumManager.addExistingFish(id, newFishMC);
    }


    static addExistingFish(id:string, mc:any):Fish {


        let newFish:Fish = new Fish(id, mc, mc.x, mc.y);


        if (!AquariumManager.fish) {
            AquariumManager.fish  = [];
        }
        AquariumManager.fish.push(newFish);


        if (AquariumManager.fishLayer) {
            AquariumManager.fishLayer.addChild(mc);
        }


        return newFish;
    }
    
    static update(frameTime:number, totalTime:number) {
        AquariumManager.fish.forEach(fish => {
            fish.update(frameTime, totalTime);
        });
    }
}

Fish.ts

import AnimatedObject from './AnimatedObject';
import Vector2 from '../physics/Vector2';
import SimplePhysicsModel from '../physics/SimplePhysicsModel';


export default class Fish extends AnimatedObject {


    public id:string;


    constructor(id, mc, x, y) {
        super(mc, x, y);
        this.id = id;
        this.physics.friction = new Vector2(0.98, 0.98);
        this.mc.gotoAndStop(0);
    }


    randomVelocity() {
        this.physics.velocity.x = -100 - (Math.random() * 200);
        this.mc.gotoAndPlay("fast");
    }


    update(frameTime:number, totalTime:number) {
        super.update(frameTime, totalTime);
        //console.log(`Fish: timeline calback:`, mc, label);
        let label = this.mc.currentLabel;
        switch (label) {
            case "idle_loop":
            case "slow_loop":
            case "fast_loop":
            case "back_loop":
                console.log(`currentLabel: ${label}`);
                this.mc.gotoAndPlay('idle');
        }
    }
}

AnimatedObject.ts

import Vector2 from '../physics/Vector2';
import SimplePhysicsModel from '../physics/SimplePhysicsModel';


export default class AnimatedObject {


    public mc:any;
    public physics:SimplePhysicsModel;
    public scale:Vector2;
    public alive:boolean;
    public randomVelocityFunction:any;
    public randomInterval:number;
    public randomIntervalId:number;


    constructor(mc, x, y, randomInterval?:number) {
        this.mc = mc;
        this.physics = new SimplePhysicsModel();
        this.physics.friction = new Vector2(0.98, 1);
        this.scale = new Vector2(1, 1);
        this.setPosition(x, y);
        this.alive = true;
        this.randomInterval = randomInterval || 3000;


        this.randomVelocityFunction = this.randomVelocity.bind(this);
        this.randomIntervalId = setInterval(this.randomVelocityFunction, this.randomInterval);
    }


    update(frameTime:number, totalTime:number) {
        this.physics.update(frameTime);
        this.wrap();
        this.mc.x = this.physics.x;
        this.mc.y = this.physics.y;
    }


    wrap() {
        if (this.physics.position.x < -220) {
            this.physics.position.x = 1280 + 36;
        }


        if (this.physics.position.x > 1500) {
            this.physics.position.x = -36;
        }
    }


    randomVelocity() {
        this.physics.velocity.x = -100 - (Math.random() * 200);
    }


    setPosition(x, y) {
        this.physics.position.x = x;
        this.physics.position.y = y;
        this.mc.x = this.physics.x;
        this.mc.y = this.physics.y;
    }
}

Questions/Comments? Feel free to join our forum discussion

Andrew Rapo
Executive Producer, Business Development & Marketing

Become a Jibo developer