notebooks/SMPL_basic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SMPL-basic\n",
    "\n",
    "SMPL is a classic body model for human body shape and pose estimation. We can get joints position and the vertices of mesh from SMPL's results.\n",
    "\n",
    "In this notebook, we will introduce the basic usage of `smplx.SMPL` model, a rough feeling of the four parameters, and some common problems you may encounter.\n",
    "\n",
    "And there is a \"SMPL family\", which is a set of models related to SMPL. You can have a overview of them in [SMPL wiki](https://meshcapade.wiki/SMPL).\n",
    "\n",
    "## Preparation\n",
    "\n",
    "Usually, we use three different checkpoints: `SMPL_NEUTRAL.pkl`, `SMPL_MALE.pkl` and `SMPL_FEMALE.pkl`. You can get the first one from [SMPLify > Downloads > \"SMPLIFY_CODE_V2.ZIP\"](https://smplify.is.tue.mpg.de/download.php) and the other two from [SMPL > Downloads > \"Download version 1.0.0 for Python 2.7 (female/male. 10 shape PCs)\"](https://smpl.is.tue.mpg.de/download.php). Or you can choose to download them through scripts like [these lines (WHAM's script for example)](https://github.com/yohanshin/WHAM/blob/2b54f7797391c94876848b905ed875b154c4a295/fetch_demo_data.sh#L2-L30).\n",
    "\n",
    "> If you download the checkpoint from the website yourself, you may get a `.pkl` file named `basicModel_neutral_lbs_10_207_0_v1.0.0.pkl` rather then `SMPL_NEUTRAL.pkl`. Don't be worried, just renamed it.\n",
    "\n",
    "> The size of `v1.0.0` version body models is usually around 37MB. Those models support only **10** principal components for shape(betas). You can also find the `v1.1.0` version of male, female and neutral body models from [SMPL > Downloads > \"Download version 1.1.0 for Python 2.7 (female/male/neutral, 300 shape PCs)\"](https://smpl.is.tue.mpg.de/download.php), they are usually in the size of 247MB and support **300** principal components for shape(betas).\n",
    "\n",
    "\n",
    "After downloading the SMPL's checkpoints, you should put them to `data_inputs/body_models/smpl`, your directory tree should look like this:\n",
    "\n",
    "```\n",
    ".\n",
    "├── SMPL_FEMALE.pkl\n",
    "├── SMPL_MALE.pkl\n",
    "└── SMPL_NEUTRAL.pkl\n",
    "```\n",
    "\n",
    "## Tutorials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Environment Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Packages you may use very often.\n",
    "import torch\n",
    "import numpy as np\n",
    "from smplx import SMPL\n",
    "from pytorch3d import transforms  # You may use this package when performing rotation representation transformation.\n",
    "\n",
    "# Things you don't need to care about. They are just for driving the tutorials.\n",
    "from lib.utils.path_manager import PathManager\n",
    "from lib.viewer.wis3d_utils import HWis3D as Wis3D\n",
    "from lib.skeleton import Skeleton_SMPL24\n",
    "\n",
    "pm = PathManager()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load SMPL model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SMPL has different models weights for different genders. Make sure you use the correct model for your project.\n",
    "\n",
    "Usually, we just use the neutral model if we can't access the gender information. \n",
    "\n",
    "Here, we will just use neutral model for simplicity. You can try the other genders if you want. You can just re-assign the `body_model` variable in the next cell and re-run the remaining cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING: You are using a SMPL model, with only 10 shape coefficients.\n",
      "WARNING: You are using a SMPL model, with only 10 shape coefficients.\n",
      "WARNING: You are using a SMPL model, with only 10 shape coefficients.\n"
     ]
    }
   ],
   "source": [
    "body_models = {}\n",
    "genders = ['neutral', 'female', 'male']  # case insensitive\n",
    "\n",
    "for gender in genders:\n",
    "    body_models[gender] = SMPL(\n",
    "            model_path = pm.inputs / 'body_models' / 'smpl',\n",
    "            gender     = gender,\n",
    "        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare some parameters for later inference.\n",
    "B = 150\n",
    "body_model : SMPL = body_models['neutral']  # use neutral for example\n",
    "\n",
    "# Prepare mesh template for later visualization.\n",
    "# Tips: mesh = vertices + faces, and the faces are the indices of vertices, which won't change across SMPL's outputs.\n",
    "mesh_temp : np.ndarray = body_model.faces  # (13776, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SMPL Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([150, 45, 3]) torch.Size([150, 6890, 3])\n"
     ]
    }
   ],
   "source": [
    "# Inference.\n",
    "smpl_out = body_model(\n",
    "        betas         = torch.zeros(B, 10),     # shape coefficients\n",
    "        global_orient = torch.zeros(B, 1, 3),   # axis-angle representation\n",
    "        body_pose     = torch.zeros(B, 23, 3),  # axis-angle representation\n",
    "        transl        = torch.zeros(B, 3),\n",
    "    )\n",
    "\n",
    "# Check output.\n",
    "joints : torch.Tensor = smpl_out.joints    # (B, 45, 3)\n",
    "verts  : torch.Tensor = smpl_out.vertices  # (B, 6890, 3)\n",
    "print(joints.shape, verts.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SMPL Skeleton\n",
    "\n",
    "SMPL has a 24-joint skeleton. We can get the joints position from the SMPL model's output.\n",
    "\n",
    "> You may notice that there are 45 joints returned by `smplx.SMPL` in total. Here we only refer to the first 24 joints as the skeleton. You can check [SMPL-details#Understand-the-Joints-45](./SMPL_details.ipynb) for more details about this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The joints' index of SMPL model is shown below. The figure is from the [SMPL-made-simple-FAQs](https://files.is.tue.mpg.de/black/talks/SMPL-made-simple-FAQs.pdf).\n",
    "\n",
    "![](./assets/SMPL-joints.png)\n",
    "\n",
    "And the bones (edge in kinematic tree) are shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chains = [\n",
    "        [ 0,  1,  4,  7, 10],      # left leg\n",
    "        [ 0,  2,  5,  8, 11],      # right leg\n",
    "        [ 0,  3,  6,  9, 12, 15],  # spine & head\n",
    "        [12, 13, 16, 18, 20, 22],  # left arm\n",
    "        [12, 14, 17, 19, 21, 23],  # right arm\n",
    "    ]\n",
    "bones = [\n",
    "        [ 0,  1], [ 1,  4], [ 4,  7], [ 7, 10],            # left leg\n",
    "        [ 0,  2], [ 2,  5], [ 5,  8], [ 8, 11],            # right leg\n",
    "        [ 0,  3], [ 3,  6], [ 6,  9], [ 9, 12], [12, 15],  # spine & head\n",
    "        [12, 13], [13, 16], [16, 18], [18, 20], [20, 22],  # left arm\n",
    "        [12, 14], [14, 17], [17, 19], [19, 21], [21, 23],  # right arm\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SMPL Parameters\n",
    "\n",
    "Here we will learn the main for SMPL parameters through several demos. The main parameters are:\n",
    "\n",
    "1. betas\n",
    "2. global_orient\n",
    "3. body_pose\n",
    "4. transl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. betas | $\\beta \\in \\R^{||\\beta||}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Betas control the shape of the model. Usually we use the default 10 shape coefficients. It depends on the model you load.\n",
    "\n",
    "You may see this(below) before, that means you are using a model with 10 shape coefficients.\n",
    "\n",
    "> \"WARNING: You are using a SMPL model, with only 10 shape coefficients.\" "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learn_betas(\n",
    "    selected_component : int = 0,\n",
    "    lower_bound : int = -5,\n",
    "    upper_bound : int = +5,\n",
    "):\n",
    "    def make_fake_data():\n",
    "        fake_betas = torch.zeros(B, 10)\n",
    "        fake_betas[:, selected_component] = torch.linspace(lower_bound, upper_bound, B)\n",
    "        return fake_betas\n",
    "    fake_betas = make_fake_data()\n",
    "\n",
    "    # Inference.\n",
    "    smpl_out = body_model(\n",
    "            betas         = fake_betas,             # shape coefficients\n",
    "            global_orient = torch.zeros(B, 1, 3),   # axis-angle representation\n",
    "            body_pose     = torch.zeros(B, 23, 3),  # axis-angle representation\n",
    "            transl        = torch.zeros(B, 3),\n",
    "        )\n",
    "\n",
    "    # Check output.\n",
    "    joints : torch.Tensor = smpl_out.joints    # (B, 45, 3)\n",
    "    verts  : torch.Tensor = smpl_out.vertices  # (B, 6890, 3)\n",
    "\n",
    "    def visualize_results():\n",
    "        \"\"\" This part is to visualize the results. You are supposed to ignore this part. \"\"\"\n",
    "        shape_wis3d = Wis3D(\n",
    "                pm.outputs / 'wis3d',\n",
    "                'SMPL-parameters-beta',\n",
    "            )\n",
    "\n",
    "        shape_wis3d.add_motion_verts(\n",
    "                verts  = verts,\n",
    "                name   = f'betas[:, {selected_component}] from {lower_bound} to {upper_bound}',\n",
    "                offset = 0,\n",
    "            )\n",
    "        shape_wis3d.add_motion_skel(\n",
    "                joints = joints[:, :24],\n",
    "                bones  = Skeleton_SMPL24.bones,\n",
    "                colors = Skeleton_SMPL24.bone_colors,\n",
    "                name   = f'betas[:, {selected_component}] from {lower_bound} to {upper_bound}',\n",
    "                offset = 0,\n",
    "            )\n",
    "    visualize_results()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will visualize the effects of the changes on certain coefficient.\n",
    "\n",
    "Here, `learn_betas(k)` means we will visualize the SMPL outputs when the k-th coefficient is changed from -5 to +5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn_betas(0)\n",
    "learn_betas(1)\n",
    "learn_betas(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now start the viewer.\n",
    "\n",
    "You will see that, the skeleton will mis-align with the mesh when the coefficients are very \"sharp\".\n",
    "Some of the coefficient control the height, the length of the limbs, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Serving on http://0.0.0.0:19090\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "# Start the server. (Remember to terminate the cell before going on.)\n",
    "!wis3d --vis_dir {pm.outputs / 'wis3d'} --host 0.0.0.0 --port 19090"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. global_orient | $\\theta_r\\in\\R^3$ (part of $\\theta \\in \\R^{3\\times24}$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Global orient control the face direction of the virtual human, which is also the \"rotation\" of the root joint.\n",
    "\n",
    "You may have to check [axis angle](https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation) before going on.\n",
    "For example, a vector $\\vec{r} = [x, y, z]$ represents a rotation around the axis $\\frac{\\vec{r}}{||\\vec{r}||}$ in radians $||\\vec{r}||$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learn_orient():\n",
    "    def make_fake_data():\n",
    "        fake_orient = torch.zeros(B, 1, 3)\n",
    "        fake_orient[   : 50, :, 0] = torch.linspace(0, 2 * np.pi, 50).reshape(50, 1)  # about x-axis\n",
    "        fake_orient[ 50:100, :, 1] = torch.linspace(0, 2 * np.pi, 50).reshape(50, 1)  # about y-axis\n",
    "        fake_orient[100:150, :, :] = torch.linspace(0, 2 * np.pi, 50).reshape(50, 1, 1).repeat(1, 1, 3)  # about x=y=z\n",
    "        return fake_orient\n",
    "    fake_orient = make_fake_data()\n",
    "\n",
    "    # Inference.\n",
    "    smpl_out = body_model(\n",
    "            betas         = torch.zeros(B, 10),     # shape coefficients\n",
    "            global_orient = fake_orient,            # axis-angle representation\n",
    "            body_pose     = torch.zeros(B, 23, 3),  # axis-angle representation\n",
    "            transl        = torch.zeros(B, 3),\n",
    "        )\n",
    "\n",
    "    # Check output.\n",
    "    joints : torch.Tensor = smpl_out.joints    # (B, 45, 3)\n",
    "    verts  : torch.Tensor = smpl_out.vertices  # (B, 6890, 3)\n",
    "\n",
    "    def visualize_results():\n",
    "        \"\"\" This part is to visualize the results. You are supposed to ignore this part. \"\"\"\n",
    "        orient_wis3d = Wis3D(\n",
    "                pm.outputs / 'wis3d',\n",
    "                'SMPL-parameters-global_orient',\n",
    "            )\n",
    "\n",
    "        # Prepare the rotation axis.\n",
    "        axis_x   = torch.tensor([[0, 0, 0], [3, 0, 0]], dtype=torch.float32)\n",
    "        axis_y   = torch.tensor([[0, 0, 0], [0, 3, 0]], dtype=torch.float32)\n",
    "        axis_xyz = torch.tensor([[0, 0, 0], [1, 1, 1]], dtype=torch.float32)\n",
    "        axis_all = torch.concat(\n",
    "            [\n",
    "                axis_x.reshape(1, 2, 3).repeat(50, 1, 1),\n",
    "                axis_y.reshape(1, 2, 3).repeat(50, 1, 1),\n",
    "                axis_xyz.reshape(1, 2, 3).repeat(50, 1, 1),\n",
    "            ],\n",
    "            dim = 0,\n",
    "        )\n",
    "        axis_all[:, :, :] += joints[:, [0], :] # move the axis to the root joints\n",
    "\n",
    "\n",
    "        orient_wis3d.add_vec_seq(\n",
    "            vecs = axis_all,\n",
    "            name = 'rotation axis',\n",
    "        )\n",
    "        orient_wis3d.add_motion_verts(\n",
    "                verts  = verts,\n",
    "                name   = f'vertices',\n",
    "                offset = 0,\n",
    "            )\n",
    "        orient_wis3d.add_motion_skel(\n",
    "                joints = joints[:, :24],\n",
    "                bones  = Skeleton_SMPL24.bones,\n",
    "                colors = Skeleton_SMPL24.bone_colors,\n",
    "                name   = f'skeleton',\n",
    "                offset = 0,\n",
    "            )\n",
    "    visualize_results()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will visualize the effects of the changes on `global_orient`.\n",
    "\n",
    "Here, `learn_orient()` will rotate the digital human in three ways:\n",
    "\n",
    "1. `fake_orient[  0: 50]` = $[0, 0, 0] \\rightarrow [2\\pi,  0, 0 ]$, rotation about $x$-axis\n",
    "2. `fake_orient[ 50:100]` = $[0, 0, 0] \\rightarrow [ 0, 2π, 0 ]$, rotation about $y$-axis\n",
    "3. `fake_orient[100:150]` = $[0, 0, 0] \\rightarrow [2π, 2π, 2π]$, rotation about $x=y=z$ axis\n",
    "\n",
    "You are supposed to make sure you understand the axis-angle representation before going on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn_orient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now start the Wis3D viewer.\n",
    "\n",
    "**You will see that, the rotation axis starts from the position of root joint, rather than the origin of the coordinates.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Serving on http://0.0.0.0:19095\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "!wis3d --vis_dir {pm.outputs / 'wis3d'} --host 0.0.0.0 --port 19090"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is still one thing you should know: the exact value of global orientation is related to the **coordinates** (e.g., camera coordinates, global coordinates) you are using. (So is the translation in SMPL.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. body_pose | $\\theta_b\\in\\R^{3\\times23}$ (part of $\\theta \\in \\R^{3\\times24}$)\n",
    "\n",
    "You should be sensitive to these numbers combinations: (23, 3), (23, 6), (23, 3, 3), (69,), (24, 3), (24, 6), (24, 3, 3), (72,). The tensor or array with these shapes are usually related to the body pose.\n",
    "\n",
    "Also, a pose is represented in the way of **kinematic chains**, the `body_pose` provide the **relative rotation of each joint to its parent joint**, and the SMPL model will solve a **forward kinematics** problem to get the final position of each joints, i.e. the final pose.\n",
    "\n",
    "> Check [this lecture (GAMES 105 Lec3)](https://www.bilibili.com/video/BV1GG4y1p7fF?p=3&vd_source=13807e82155f985591ed6f1b4e3434ed) if you are interested in the topic of forward/inverse kinematic.\n",
    "\n",
    "Sometimes we will group the `global_orient` and `body_pose` together as a 24 \"joints\" `pose`, and the `global_orient` is always the first element of this `pose`.\n",
    "\n",
    "Although in SMPL, the joint rotation is represented in the form of **axis-angle**, people are more likely to use an **6D-rotation** representation extracted from the 3x3 rotation matrix for a network to train. We will dive into this problem in another notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. transl | $\\Gamma\\in\\R^{3}$\n",
    "\n",
    "Translation control the position of the virtual human in the 3D space. In camera coordinates, the translation usually represent the distance between the camera and the human. In global coordinates, the translation usually has the similar meaning as the movement of the human.\n",
    "\n",
    "We also use the word \"trajectory\" to represent the historical position of the root joint. Sometimes, the \"trajectory\" will be projected to the ground plane. Remember, the specific meaning of the translation is related to the specific work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learn_transl(rotation:bool = False):\n",
    "    def make_fake_data():\n",
    "        phase = torch.arange(50) / 50.0 * (2 * np.pi)  # 0 ~ 2𝜋\n",
    "\n",
    "        # Generate fake translation.\n",
    "        fake_transl = torch.zeros(B, 3)\n",
    "        # Part 1, [0, 50)\n",
    "        fake_transl[   : 25, 2] = torch.sin(phase[:25])  # along z-axis\n",
    "        fake_transl[ 25: 50, 1] = torch.sin(phase[:25])  # along y-axis\n",
    "        # Part 2, [50, 75) + [75, 100)\n",
    "        fake_transl[ 50:100, 1] = torch.sin(phase)       # along y-axis\n",
    "        fake_transl[ 50: 75, 0] = torch.sin(phase[::2])  # along y-axis\n",
    "        fake_transl[ 75:100, 2] = torch.sin(phase[::2])  # along y-axis\n",
    "        # Part 3, [100, 150)\n",
    "        fake_transl[100:150, 0] = torch.cos(phase) * phase / (2 * np.pi)\n",
    "        fake_transl[100:150, 2] = torch.sin(phase) * phase / (2 * np.pi)\n",
    "\n",
    "        # Generate fake rotation (if needed).\n",
    "        fake_orient = torch.zeros(B, 1, 3)\n",
    "        if rotation:\n",
    "            fake_orient[:, :, 1] = torch.linspace(0, 3 * (2 * np.pi), B).reshape(B, 1)  # about y-axis\n",
    "\n",
    "        return fake_transl, fake_orient\n",
    "\n",
    "    fake_transl, fake_orient = make_fake_data()\n",
    "\n",
    "    # Inference.\n",
    "    smpl_out = body_model(\n",
    "            betas         = torch.zeros(B, 10),     # shape coefficients\n",
    "            global_orient = fake_orient,            # axis-angle representation\n",
    "            body_pose     = torch.zeros(B, 23, 3),  # axis-angle representation\n",
    "            transl        = fake_transl,\n",
    "        )\n",
    "\n",
    "    # Check output.\n",
    "    joints : torch.Tensor = smpl_out.joints    # (B, 45, 3)\n",
    "    verts  : torch.Tensor = smpl_out.vertices  # (B, 6890, 3)\n",
    "\n",
    "    def visualize_results():\n",
    "        \"\"\" This part is to visualize the results. You are supposed to ignore this part. \"\"\"\n",
    "        transl_wis3d = Wis3D(\n",
    "                pm.outputs / 'wis3d',\n",
    "                'SMPL-parameters-transl',\n",
    "            )\n",
    "\n",
    "        transl_wis3d.add_traj(\n",
    "                positions = fake_transl,\n",
    "                name      = f'trajectory (rotating)' if rotation else 'trajectory',\n",
    "                offset    = 0,\n",
    "            )\n",
    "        transl_wis3d.add_motion_verts(\n",
    "                verts  = verts,\n",
    "                name   = f'vertices (rotating)' if rotation else 'vertices',\n",
    "                offset = 0,\n",
    "            )\n",
    "        transl_wis3d.add_motion_skel(\n",
    "                joints = joints[:, :24],\n",
    "                bones  = Skeleton_SMPL24.bones,\n",
    "                colors = Skeleton_SMPL24.bone_colors,\n",
    "                name   = f'skeleton (rotating)' if rotation else 'skeleton',\n",
    "                offset = 0,\n",
    "            )\n",
    "    visualize_results()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will visualize the effects of the changes on `transl`.\n",
    "\n",
    "Here, `learn_transl()` will make the digital human moves. Please check the code yourself and match the lines with the movements in the visualization results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn_transl(rotation=False)\n",
    "learn_transl(rotation=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now start the Wis3D viewer.\n",
    "\n",
    "There are two things you should notice:\n",
    "\n",
    "1. The `transl` is defined in a static coordinate (compared to the ego coordinate of the agent). So the orientation changes will not affect the translation. Check the visualization results, you will find the trajectory of the rotating human and the non-rotating human are the same.\n",
    "2. The position of the root joint has small differences with the `transl` in the SMPL model, as the root joint won't be put on the origin of the coordinates in zero-pose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Serving on http://0.0.0.0:19090\n",
      "^C\n"
     ]
    }
   ],
   "source": [
    "!wis3d --vis_dir {pm.outputs / 'wis3d'} --host 0.0.0.0 --port 19090"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "- [SMPL Project Page](https://smpl.is.tue.mpg.de/)\n",
    "- [SMPL-made-simple-FAQs](https://files.is.tue.mpg.de/black/talks/SMPL-made-simple-FAQs.pdf)\n",
    "- [SMPL wiki](https://meshcapade.wiki/SMPL)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "lh",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}