Skip to Content
←All Articles

Handle Video Call Invitations with Agora RTM and RTC in your Vuejs and Flask Application

 — #Agora#Vuejs#Flask

Real-time communication on the web has grown in leaps and bounds with the improvement of web technologies like WebRTC and accelerated by the need for virtual meetings given the pandemic we are facing.

If you have ever wondered how to include a video call feature to your web application with a similar experience we have in most mobile apps, you found the right article. Think of it as a Video Call feature inside WhatsApp Web. We will implement this fully with Agora's RTM and RTC SDKs.

Features to Expect in the Article

  • Inviting a user to a video call with an incoming call notification.
  • Returning the appropriate feedback if they are offline.
  • The caller can terminate the call before it is answered
  • The recipient/callee can decline/reject the call
  • The recipient/callee can accept/receive the call for the video call communication to be established.

Prerequisites

You can use the following starter project: Flask Auth Starter Project. We are going to build on top of this starter project.

Project Setup

  1. Create and activate a python3 virtual environment for this project.
  2. Open your terminal or command prompt and navigate to the starter project you downloaded as part of the prerequisites. The folder is named agora-flask-starter.
  3. Follow the instructions in the README.md file in the agora-flask starter to set up the application.
  4. Download the AgoraDynamicKey Python3 code from the Agora repository: AgoraDynamicKey
  5. Download the latest version of the Agora RTM SDK. Click here to download

Configuring the Backend

We will create a new app named agora_rtm, register its blueprint, create the needed static, templates and views.

  1. Create the folders needed for the app.
#!/bin/bash
mkdir app/agora_rtm
mkdir app/static/agora_rtm
mkdir app/templates/agora_rtm
  1. Create the Agora view.
  • Create views.py and init.py inside app/agora_rtm directory from your terminal.
#!/bin/bash
touch app/agora_rtm/views.py
touch app/agora_rtm/__init__.py
  • Add the following to init.py
from flask import Blueprint
agora_rtm = Blueprint('agora_rtm', '__init__')

from . import views  # isort:skip
  • Add the following to the views.py
import os
import time
from flask import render_template, jsonify, request
from flask_login import login_required, current_user

from . import agora_rtm
from ..models import User
from .agora_key.RtcTokenBuilder import RtcTokenBuilder, Role_Attendee
from .agora_key.RtmTokenBuilder import RtmTokenBuilder, Role_Rtm_User


@agora_rtm.route('/agora-rtm')
@login_required
def index():
    users = User.query.all()
    all_users = [user.to_json() for user in users]
    return render_template('agora_rtm/index.html', title='Agora Video Call with RTM', allUsers=all_users, agoraAppID=os.environ.get('AGORA_APP_ID'))


@agora_rtm.route('/users')
def fetch_users():
    users = User.query.all()
    all_users = [user.to_json() for user in users]
    return jsonify(all_users)


@agora_rtm.route('/agora-rtm/token',  methods=['POST'])
def generate_agora_token():
    auth_user = current_user.to_json()
    appID = os.environ.get('AGORA_APP_ID')
    appCertificate = os.environ.get('AGORA_APP_CERTIFICATE')
    channelName = request.json['channelName']
    userAccount = auth_user['username']
    expireTimeInSeconds = 3600
    currentTimestamp = int(time.time())
    privilegeExpiredTs = currentTimestamp + expireTimeInSeconds

    token = RtcTokenBuilder.buildTokenWithAccount(
        appID, appCertificate, channelName, userAccount, Role_Attendee, privilegeExpiredTs)

    rtm_token = RtmTokenBuilder.buildToken(
        appID, appCertificate, userAccount, Role_Rtm_User, privilegeExpiredTs)
    return jsonify({'token': token, 'rtm_token': rtm_token, 'appID': appID})
  1. Register the agora_rtm blueprint.

Import the agora_rtm app and register as a blueprint inside app/init.py

from .agora_rtm import agora_rtm as agora_rtm_blueprint
app.register_blueprint(agora_rtm_blueprint)

Place the above code before the return app statement.

  1. Add the downloaded AgoraDynamicKey generator files.
  • Create a subdirectory named agora_key inside app/agora_rtm directory.
#!/bin/bash
mkdir app/agora_rtm/agora_key
  • Copy AccessToken.py and RtcTokenBuilder.py from the src directory in the downloaded files and add them to the agora_key directory.

Breakdown of Methods in agora_rtm/views.py

index: To view the video call page. Only authenticated users can view the page. Non-authenticated users are redirected to the login page. We return a list of all the registered users.

fetch_users: Returns all the registered users as a json response.

generate_agora_token: Returns the token used for the RTM and RTC connections.

token: The RTC token

rtm_token: The RTM token

Note that we have set the token's expiration time to 3600s i.e 1 hr. You can modify the endpoint to use the expiration time you want.

Configuring the Front End

We will create the user interface for making and receiving the video call with the ability to toggle the on and off states of the camera and the microphone. We show incoming call notifications where the recipient can accept or reject the call.

  1. Add the Downloaded RTM SDK to the project.

    • Unzip the file we downlooaded in Project Setup (5).
    • Navigate to the libs folder and copy the agora-rtm-sdk-1.4.4.js file into static/agora_rtm
    • Rename the copied file to agora-rtm.js

  2. Add the HTML file for the index view

    The HTML file will contain the links to the CDN for Agora RTC SDK, Agora RTM SDK, Vue.js, Axios, Bootstrap for styling, and our custom CSS and JavaScript.

The index.html file will also inherit a base template, which is used to render the view.

* Create an **index.html** file inside **templates/agora_rtm**
#!/bin/bash
touch app/templates/agora_rtm/index.html
* Add the following code to the **index.html** file.
{% extends "base.html" %} {% block head_scripts %}
<link
  rel="stylesheet"
  type="text/css"
  href="{{ url_for('static', filename='agora/index.css') }}"
/>
<script src="https://download.agora.io/sdk/release/AgoraRTC_N-4.7.0.js"></script>
<script src="{{ url_for('static', filename='agora_rtm/agora-rtm.js') }}"></script>
{% endblock head_scripts %} {% block content%}
<div id="app">
  <div class="container my-5">
    <div class="row">
      <div class="col" v-if="isLoggedIn">
        <div class="btn-group" role="group" id="btnGroup">
          {% for singleUser in allUsers%} {% if singleUser['id'] !=
          current_user['id'] %} {% set username = singleUser['username']%}
          <button
            type="button"
            class="btn btn-primary mr-2 my-2"
            @click="placeCall('{{username}}')"
          >
            Call {{ username}}
            <span class="badge badge-light"
              >${updatedOnlineStatus?.["{{username}}"]?.toLowerCase() ||
              'offline'}</span
            >
          </button>
          {% endif %} {% endfor %}
        </div>
      </div>
    </div>

    <div class="row my-5" v-if="isCallingUser">
      <div class="col-12">
        <p>${callingUserNotification}</p>
        <button type="button" class="btn btn-danger" @click="cancelCall">
          Cancel Call
        </button>
      </div>
    </div>

    <!-- Incoming Call  -->
    <div class="row my-5" v-if="incomingCall">
      <div class="col-12">
        <!-- <p>Incoming Call From <strong>${ incomingCaller }</strong></p> -->
        <p>${incomingCallNotification}</p>
        <div class="btn-group" role="group">
          <button
            type="button"
            class="btn btn-danger"
            data-dismiss="modal"
            @click="declineCall"
          >
            Decline
          </button>
          <button
            type="button"
            class="btn btn-success ml-5"
            @click="acceptCall"
          >
            Accept
          </button>
        </div>
      </div>
    </div>
    <!-- End of Incoming Call  -->
  </div>

  <section id="video-container" v-if="callPlaced">
    <div id="local-video" ref="localVideo"></div>
    <div id="remote-video" ref="remoteVideo"></div>

    <div class="action-btns">
      <button type="button" class="btn btn-info" @click="handleAudioToggle">
        ${ mutedAudio ? "Unmute" : "Mute" }
      </button>
      <button
        type="button"
        class="btn btn-primary mx-4"
        @click="handleVideoToggle"
      >
        ${ mutedVideo ? "ShowVideo" : "HideVideo" }
      </button>
      <button type="button" class="btn btn-danger" @click="endCall">
        EndCall
      </button>
    </div>
  </section>
</div>
{% endblock content %}

<!-- Add Scripts -->
{% block bottom_scripts%}
<script>
  const AUTH_USER = "{{current_user['username']}}";
  const AUTH_USER_ID = "{{current_user['id']}}";
  const CSRF_TOKEN = "{{ csrf_token }}";
  const AGORA_APP_ID = "{{agoraAppID}}";
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.21.1/axios.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/vue/dist/vue.js"></script>
<script src="{{ url_for('static', filename='agora_rtm/index.js') }}"></script>
{% endblock bottom_scripts %}

We use Flask’s templating language to be able to reuse some code. As indicated earlier, we inherit a base template named base.html. It has the following blocks:

  • head_scripts: This is the block where we place the link to the Agora RTC and RTM SDKs and our index.css for styling the video call page.
  • content: The content block contains the user interface for rendering the video stream with its control buttons.
  • bottom_scripts: This block contains the CDN links to Axios for sending AJAX requests, and Vue.js for writing the client-side logic for our video chat application. We also have index.js for our custom JavaScript code.
  1. Create the static files

We have index.css for custom styling and index.js, which is our script for handling the call logic. Run the following command to create the files from your terminal or command prompt:

#!/bin/bash
touch app/static/agora_rtm/index.js
touch app/static/agora_rtm/index.css

Add the following to index.js:

const app = new Vue({
  el: "#app",
  delimiters: ["${", "}"],
  data: {
    callPlaced: false,
    localStream: null,
    mutedAudio: false,
    mutedVideo: false,
    onlineUsers: [],
    isLoggedIn: false,
    incomingCall: false,
    incomingCaller: "",
    incomingCallNotification: "",
    rtmClient: null,
    rtmChannel: null,
    rtcClient: null,
    users: [],
    updatedOnlineStatus: {},
    channelName: null,
    isCallingUser: false,
    callingUserNotification: "",
    localAudioTrack: null,
    localVideoTrack: null,
    remoteVideoTrack: null,
    remoteAudioTrack: null,
  },
  mounted() {
    this.fetchUsers();
    this.initRtmInstance();
  },

  created() {
    window.addEventListener("beforeunload", this.logoutUser);
  },

  beforeDestroy() {
    this.endCall();
    this.logoutUser();
  },

  computed: {
    canDisplayUsers() {
      console.log(Object.keys(this.updatedOnlineStatus).length);
      console.log(Object.keys(this.users).length);

      return (
        Object.keys(this.updatedOnlineStatus).length ===
        Object.keys(this.users).length - 1
      );
    },
  },

  methods: {
    async fetchUsers() {
      const { data } = await axios.get("/users");
      this.users = data;
    },

    async logoutUser() {
      console.log("destroyed!!!");
      this.rtmChannel.leave(AUTH_USER);
      await this.rtmClient.logout();
    },

    async initRtmInstance() {
      // initialize an Agora RTM instance
      this.rtmClient = AgoraRTM.createInstance(AGORA_APP_ID, {
        enableLogUpload: false,
      });

      // RTM Channel to be used
      this.channelName = "videoCallChannel";

      // Generate the RTM token
      const { data } = await this.generateToken(this.channelName);

      // Login when it mounts
      await this.rtmClient.login({
        uid: AUTH_USER,
        token: data.rtm_token,
      });

      this.isLoggedIn = true;

      // RTM Message Listeners
      this.rtmClient.on("MessageFromPeer", (message, peerId) => {
        console.log("MessageFromPeer");
        console.log("message: ", message);
        console.log("peerId: ", peerId);
      });

      // Display connection state changes
      this.rtmClient.on("ConnectionStateChanged", (state, reason) => {
        console.log("ConnectionStateChanged");
        console.log("state: ", state);
        console.log("reason: ", reason);
      });
      // Emitted when a Call Invitation is sent from Remote User
      this.rtmClient.on("RemoteInvitationReceived", (data) => {
        this.remoteInvitation = data;
        this.incomingCall = true;
        this.incomingCaller = data.callerId;
        this.incomingCallNotification = `Incoming Call From ${data.callerId}`;

        data.on("RemoteInvitationCanceled", () => {
          console.log("RemoteInvitationCanceled: ");
          this.incomingCallNotification = "Call has been cancelled";
          setTimeout(() => {
            this.incomingCall = false;
          }, 5000);
        });
        data.on("RemoteInvitationAccepted", (data) => {
          console.log("REMOTE INVITATION ACCEPTED: ", data);
        });
        data.on("RemoteInvitationRefused", (data) => {
          console.log("REMOTE INVITATION REFUSED: ", data);
        });
        data.on("RemoteInvitationFailure", (data) => {
          console.log("REMOTE INVITATION FAILURE: ", data);
        });
      });

      // Subscribes to the online statuses of all users apart from
      // the currently authenticated user
      this.rtmClient.subscribePeersOnlineStatus(
        this.users
          .map((user) => user.username)
          .filter((user) => user !== AUTH_USER)
      );

      this.rtmClient.on("PeersOnlineStatusChanged", (data) => {
        this.updatedOnlineStatus = data;
      });

      // Create a channel and listen to messages
      this.rtmChannel = this.rtmClient.createChannel(this.channelName);

      // Join the RTM Channel
      this.rtmChannel.join();

      this.rtmChannel.on("ChannelMessage", (message, memberId) => {
        console.log("ChannelMessage");
        console.log("message: ", message);
        console.log("memberId: ", memberId);
      });

      this.rtmChannel.on("MemberJoined", (memberId) => {
        console.log("MemberJoined");

        // check whether user exists before you add them to the online user list
        const joiningUserIndex = this.onlineUsers.findIndex(
          (member) => member === memberId
        );
        if (joiningUserIndex < 0) {
          this.onlineUsers.push(memberId);
        }
      });

      this.rtmChannel.on("MemberLeft", (memberId) => {
        console.log("MemberLeft");
        console.log("memberId: ", memberId);
        const leavingUserIndex = this.onlineUsers.findIndex(
          (member) => member === memberId
        );
        this.onlineUsers.splice(leavingUserIndex, 1);
      });

      this.rtmChannel.on("MemberCountUpdated", (data) => {
        console.log("MemberCountUpdated");
      });
    },

    async placeCall(calleeName) {
      // Get the online status of the user.
      // For our use case, if the user is not online we cannot place a call.
      // We send a notification to the caller accordingly.
      this.isCallingUser = true;

      this.callingUserNotification = `Calling ${calleeName}...`;
      const onlineStatus = await this.rtmClient.queryPeersOnlineStatus([
        calleeName,
      ]);

      if (!onlineStatus[calleeName]) {
        setTimeout(() => {
          this.callingUserNotification = `${calleeName} could not be reached`;

          setTimeout(() => {
            this.isCallingUser = false;
          }, 5000);
        }, 5000);
      } else {
        // Create a channel/room name for the video call
        const videoChannelName = `${AUTH_USER}_${calleeName}`;
        // Create LocalInvitation
        this.localInvitation = this.rtmClient.createLocalInvitation(calleeName);

        this.localInvitation.on(
          "LocalInvitationAccepted",
          async (invitationData) => {
            console.log("LOCAL INVITATION ACCEPTED: ", invitationData);

            // Generate an RTM token using the channel/room name
            const { data } = await this.generateToken(videoChannelName);
            // Initialize the agora RTM Client
            this.initializeAgora();
            // Join a room using the channel name. The callee will also join the room then accept the call
            await this.joinRoom(AGORA_APP_ID, data.token, videoChannelName);
            this.isCallingUser = false;
            this.callingUserNotification = "";
          }
        );

        this.localInvitation.on("LocalInvitationCanceled", (data) => {
          console.log("LOCAL INVITATION CANCELED: ", data);
          this.callingUserNotification = `${calleeName} cancelled the call`;
          setTimeout(() => {
            this.isCallingUser = false;
          }, 5000);
        });
        this.localInvitation.on("LocalInvitationRefused", (data) => {
          console.log("LOCAL INVITATION REFUSED: ", data);
          this.callingUserNotification = `${calleeName} refused the call`;
          setTimeout(() => {
            this.isCallingUser = false;
          }, 5000);
        });

        this.localInvitation.on("LocalInvitationReceivedByPeer", (data) => {
          console.log("LOCAL INVITATION RECEIVED BY PEER: ", data);
        });

        this.localInvitation.on("LocalInvitationFailure", (data) => {
          console.log("LOCAL INVITATION FAILURE: ", data);
          this.callingUserNotification = "Call failed. Try Again";
        });

        // set the channelId
        this.localInvitation.channelId = videoChannelName;

        // Send call invitation
        this.localInvitation.send();

      }
    },

    async cancelCall() {
      await this.localInvitation.cancel();
      this.isCallingUser = false;
    },

    async acceptCall() {
      // Generate RTC token using the channelId of the caller
      const { data } = await this.generateToken(
        this.remoteInvitation.channelId
      );

      // Initialize AgoraRTC Client
      this.initializeAgora();

      // Join the room created by the caller
      await this.joinRoom(
        AGORA_APP_ID,
        data.token,
        this.remoteInvitation.channelId
      );

      // Accept Call Invitation
      this.remoteInvitation.accept();
      this.incomingCall = false;
      this.callPlaced = true;
    },

    declineCall() {
      this.remoteInvitation.refuse();
      this.incomingCall = false;
    },

    async generateToken(channelName) {
      return await axios.post(
        "/agora-rtm/token",
        {
          channelName,
        },
        {
          headers: {
            "Content-Type": "application/json",
            "X-CSRFToken": CSRF_TOKEN,
          },
        }
      );
    },

    /**
     * Agora Events and Listeners
     */
    initializeAgora() {
      this.rtcClient = AgoraRTC.createClient({ mode: "rtc", codec: "vp8" });
    },

    async joinRoom(appID, token, channel) {
      try {
        await this.rtcClient.join(appID, channel, token, AUTH_USER);
        this.callPlaced = true;
        this.createLocalStream();
        this.initializedAgoraListeners();
      } catch (error) {
        console.log(error);
      }
    },

    initializedAgoraListeners() {
      //   Register event listeners
      this.rtcClient.on("user-published", async (user, mediaType) => {
        await this.rtcClient.subscribe(user, mediaType);

        // If the remote user publishes a video track.
        if (mediaType === "video") {
          // Get the RemoteVideoTrack object in the AgoraRTCRemoteUser object.
          this.remoteVideoTrack = user.videoTrack;
          this.remoteVideoTrack.play("remote-video");
        }
        // If the remote user publishes an audio track.
        if (mediaType === "audio") {
          // Get the RemoteAudioTrack object in the AgoraRTCRemoteUser object.term
          this.remoteAudioTrack = user.audioTrack;
          // Play the remote audio track. No need to pass any DOM element.
          this.remoteAudioTrack.play();
        }
      });

      this.rtcClient.on("user-unpublished", (data) => {
        console.log("USER UNPUBLISHED: ", data);
        // await this.endCall();
      });
    },

    async createLocalStream() {
      const [microphoneTrack, cameraTrack] =
        await AgoraRTC.createMicrophoneAndCameraTracks();
      await this.rtcClient.publish([microphoneTrack, cameraTrack]);
      cameraTrack.play("local-video");
      this.localAudioTrack = microphoneTrack;
      this.localVideoTrack = cameraTrack;
    },

    async endCall() {
      this.localAudioTrack.close();
      this.localVideoTrack.close();
      this.localAudioTrack.removeAllListeners();
      this.localVideoTrack.removeAllListeners();
      await this.rtcClient.unpublish();
      await this.rtcClient.leave();
      this.callPlaced = false;
    },

    async handleAudioToggle() {
      if (this.mutedAudio) {
        await this.localAudioTrack.setMuted(!this.mutedAudio);
        this.mutedAudio = false;
      } else {
        await this.localAudioTrack.setMuted(!this.mutedAudio);
        this.mutedAudio = true;
      }
    },

    async handleVideoToggle() {
      if (this.mutedVideo) {
        await this.localVideoTrack.setMuted(!this.mutedVideo);
        this.mutedVideo = false;
      } else {
        await this.localVideoTrack.setMuted(!this.mutedVideo);
        this.mutedVideo = true;
      }
    },
  },
});

Add the following to index.css:

#video-container {
  width: 700px;
  height: 500px;
  max-width: 90vw;
  max-height: 50vh;
  margin: 0 auto;
  border: 1px solid #099dfd;
  position: relative;
  box-shadow: 1px 1px 11px #9e9e9e;
  background-color: #fff;
}

#local-video {
  width: 30%;
  height: 30%;
  position: absolute;
  left: 10px;
  bottom: 10px;
  border: 1px solid #fff;
  border-radius: 6px;
  z-index: 2;
  cursor: pointer;
}

#remote-video {
  width: 100%;
  height: 100%;
  position: absolute;
  left: 0;
  right: 0;
  bottom: 0;
  top: 0;
  z-index: 1;
  margin: 0;
  padding: 0;
  cursor: pointer;
}

.action-btns {
  position: absolute;
  bottom: 20px;
  left: 50%;
  margin-left: -50px;
  z-index: 3;
  display: flex;
  flex-direction: row;
  flex-wrap: wrap;
}

#btnGroup {
  flex-wrap: wrap;
}

Breakdown of the Agora Call Page

On the video call page (app/templates/agora_rtm/index.html), we display buttons that bear the name of each registered user and whether they are online or offline at the moment.

Placing a call:

Click on the button of the user you want to call. You see an outgoing call interface with the ability to cancel the call

Outgoing Call Notification

The callee/recipient gets an incoming call notification where they can either decline or accept the call.

Incoming call notification

Technical Explanation Video

The following Video explains the logic for the video call. Technical Details - Handle Video Call Invitations with Agora RTM and RTC in your Vuejs and Flask App

  1. Set the environment variables in .flaskenv
FLASK_APP=app.py
FLASK_ENV=development
SECRET_KEY=
SQLALCHEMY_DATABASE_URI=sqlite:///db.sqlite
SQLALCHEMY_TRACK_MODIFICATIONS=False
TEMPLATES_AUTO_RELOAD=True
SEND_FILE_MAX_AGE_DEFAULT=0
ENABLE_DEBUG=True
AGORA_APP_ID=
AGORA_APP_CERTIFICATE=

Testing

  1. Start the Flask development server from your terminal.
#!/bin/bash
flask run

  1. Open two different browsers or two instances of the same browser, with one instance in incognito mode, and navigate to the registration page: http://127.0.0.1:5000/register.

  2. In one of the browsers create four users by registering four times.

  3. Login with the account details you just created on each of the browsers from the login page: http://127.0.0.1:5000/login.

  4. Navigate to http://127.0.0.1:5000/agora_rtm

  5. In each of the browsers you opened, the other users registered on the application are displayed.

  6. In one browser, you can call an online user by clicking the button that bears their name.

  7. The other user is prompted to click the Accept button to fully establish the call.

Video Demonstration of the Video Call

To confirm that your demo is functioning properly, see my demo video as an example of how the finished project should look and function:

Demo - Handle Video Call Invitations with Agora RTM and RTC in your Vuejs and Flask App

Conclusion

The Agora RTM and RTC SDKs give you the ability to build a fully-featured video call application. You can even use the RTM SDK to implement an in-app messaging feature too.

When testing, one thing that stood out for me was the reconnection of the call when the internet connectivity on either side of the call failed for a short while.

Online Demo link: https://watermark-remover.herokuapp.com/auth/login?next=%2Fagora_rtm

Completed Project Repository (it is located on the branch named completed in the starter kit): https://github.com/Mupati/agora-flask-starter/tree/completed

Make sure the demo link or production version is served over HTTPS and the route it /agora_rtm

Test accounts:
foo@example.com: DY6m7feJtbnx3ud
bar@example.com: Me3tm5reQpWcn3Q

Other Resources

And I invite you to join the Agora Developer Slack Community