{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling time-series data\n",
    "\n",
    "Before processing any data, we need to handle all data we have: select specific subsets of these data and gather them. In practice, time data come with multiple sampling rates and time intervals. Data of one channel can be split in multiple files.\n",
    "\n",
    "Razorback provides tools to help in the task of isolating and grouping the data of interest. See the [signalset](../api/razorback.signalset.rst) module for the detailled documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up\n",
    "\n",
    "We first import razorback:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import razorback as rb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Usualy, data are read from files, in which case the [io](../api/razorback.io.rst) module is here to help. But for this tutorial, we will work with fake signals. The following (generator) function will build them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def build_fake_signals(infos):\n",
    "    \"\"\" yield some fake signal sets\n",
    "\n",
    "    Each signal set contains 5 channels (Ex, Ey, Hx, Hy, Hz)\n",
    "    for one run at one site, at a given sampling rate and given time interval.\n",
    "\n",
    "    The raw data of the signals are 'empty' arrays.\n",
    "    \"\"\"\n",
    "    tags_tpl = {'Ex_%s': 0, 'Ey_%s': 1, 'Hx_%s': 2, 'Hy_%s': 3, 'Hz_%s': 4,\n",
    "                'E_%s': (0, 1), 'H_%s': (2, 3, 4), 'site_%s': (0, 1, 2, 3, 4)}\n",
    "\n",
    "    for site_id, rate, start, stop in infos:\n",
    "        tags = {k % site_id: v for k, v in tags_tpl.items()}\n",
    "        size = int((stop-start) * rate) + 1\n",
    "        raw_data = np.empty((5, size))\n",
    "        signals = rb.SyncSignal(raw_data, rate, start)\n",
    "        yield rb.SignalSet(tags, signals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We consider a toy situation where we have recorded MT signals (Ex, Ey, Hx, Hy, Hz) on 5 sites (1, 2, 3, 4, 5), some with several runs and different sampling rates. The situation could be pictured by:\n",
    "\n",
    "```\n",
    "==== ======================================\n",
    "site   time\n",
    "==== ======================================\n",
    "  1            ~~~~~~~~\n",
    "  2    ~~~~~~   ~~~~~~      ^^^^^^^\n",
    "  3     ~~~~~~~~~~~~~~~~~  ^^^^^^^^^^\n",
    "  4    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
    "  5    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "==== ======================================\n",
    "\n",
    "~~~~~ : continuous run sampled at 512 Hz\n",
    "^^^^^ : continuous run sampled at 1024 Hz\n",
    "```\n",
    "\n",
    "We use our `build_fake_signals()` function to generate the signals and store the 8 signals in the `all_signals` list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "infos = [\n",
    "    # (site, sampling_rate, start_time, end_time)\n",
    "    (     1,           512,        110,      250),\n",
    "    (     2,           512,          0,      100),\n",
    "    (     2,           512,        120,      220),\n",
    "    (     2,          1024,        400,      450),\n",
    "    (     3,           512,         10,      300),\n",
    "    (     3,          1024,        350,      500),\n",
    "    (     4,           512,          5,      530),\n",
    "    (     5,          1024,          0,      550),\n",
    "]\n",
    "all_signals = list(build_fake_signals(infos))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We pick the four first signals that correspond to site 1 and 2 for further investigations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "signal_1, signal_2a, signal_2b, signal_2c = all_signals[:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspecting a signal set\n",
    "\n",
    "We can print signal_1 to get a summary of what we know about it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 1 run\n",
      "tags: {'Ex_1': (0,), 'Ey_1': (1,), 'Hx_1': (2,), \n",
      "       'Hy_1': (3,), 'Hz_1': (4,), 'E_1': (0, 1), \n",
      "       'H_1': (2, 3, 4), 'site_1': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:01:50  1970-01-01 00:04:10\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a [SignalSet](../api/razorback.signalset.rst#razorback.signalset.SignalSet) object that contains the records of 5 channels on 1 run. Next we see the *tags* as a dictionnary, then some data about the run.\n",
    "\n",
    "The tags are stored in a [Tags](../api/razorback.signalset.rst#razorback.signalset.Tags) object accessible by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tags(5, Ex_1=(0,), Ey_1=(1,), Hx_1=(2,), Hy_1=(3,), Hz_1=(4,), E_1=(0, 1), H_1=(2, 3, 4), site_1=(0, 1, 2, 3, 4))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.tags"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tags allows accessing to specific channels by using name identifiers. For instance, 'Ex_1' and 'Ey_1' correspond to indices 0 and 1. The tags also exposes names for groups of indices, like 'E_1' here that corresponds to the group (0, 1) i.e. the both channels 'Ex_1' and 'Ey_1'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we see the run: its sampling rate, its start time and its stop time. The run itself is accessible by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(SyncSignal([5x71681], sampling_rate=5.1e+02, start=1.1e+02, calibrations=[...]),)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.signals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actualy, we get a tuple because a `SignalSet` object can handle multiple runs. A more detailled view is given by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SyncSignal\n",
      "  - nb of channels :   5\n",
      "  - signal size    :   71681\n",
      "  - sampling rate  :   512.0 Hz\n",
      "  - start          :   1970-01-01 00:01:50\n",
      "  - stop           :   1970-01-01 00:04:10\n"
     ]
    }
   ],
   "source": [
    "print(signal_1.signals[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a [SyncSignal](../api/razorback.signalset.rst#razorback.signalset.SyncSignal) object that contains 5 channels, sampled at 512 Hz. `SyncSignal` objects are a light wrapper around the raw data of a bunch of synchronous signals. Here, synchronous means that all signals start and stop at the same time and have the same sampling rate. Note that the channels of a `SyncSignal` object are only identified by their indices not by *tags*.\n",
    "\n",
    "We can access to all the informations of a signal set through its attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.nb_channels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.nb_runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([512.])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.sampling_rates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[110., 250.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.intervals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([71681])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_1.sizes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Joining successive runs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`signal_2a`, `signal_2b` and `signal_2c` are records of 3 different runs on the site 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 1 run\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:01:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 1 run\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 1 run\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that they all have the same tags and that the 3 time intervals don’t overlap. For these reasons, we can join them (see [SignalSet.join()](../api/razorback.signalset.rst#razorback.signalset.SignalSet.join)) into one signal set, `signal_2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "signal_2 = signal_2a | signal_2b | signal_2c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we print `signal_2`, we see that now the 3 runs of site 2 are gathered into one `SignalSet` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 3 runs\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now attributes relatives to the runs give the values for all runs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 512.,  512., 1024.])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_2.sampling_rates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0., 100.],\n",
       "       [120., 220.],\n",
       "       [400., 450.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_2.intervals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0., 120., 400.])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_2.starts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that these values are passed as numpy array, allowing for a lot of operations on them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False,  True])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "signal_2.sampling_rates == 1024"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extracting time intervals\n",
    "\n",
    "The [SignalSet.extract_t()](../api/razorback.signalset.rst#razorback.signalset.SignalSet.extract_t) method can narrow the time interval of a signal set. The runs outside the interval are skipped and the others are eventually reduced to fit the interval:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 2 runs\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:50  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:20\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.extract_t(50, 200))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `SignalSet.extract_t()` method can also be used to exclude a given time interval:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 3 runs\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:00:50\n",
      "       512  1970-01-01 00:03:20  1970-01-01 00:03:40\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.extract_t(50, 200, exclude=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting and merging channels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can extract the Electric field only from the Signal Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 2 channels, 3 runs\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'E_2': (0, 1)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.E_2) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can Extracting only one component of E and H and joining them in a SignalSet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 2 channels, 3 runs\n",
      "tags: {'Ex_2': (0,), 'Hy_2': (1,)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.Ex_2 & signal_2.Hy_2) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 10 channels, 1 run\n",
      "tags: {'Ex_1': (0,), 'Ex_2': (5,), 'Ey_1': (1,), \n",
      "       'Ey_2': (6,), 'Hx_1': (2,), 'Hx_2': (7,), \n",
      "       'Hy_1': (3,), 'Hy_2': (8,), 'Hz_1': (4,), \n",
      "       'Hz_2': (9,), 'E_1': (0, 1), 'E_2': (5, 6), \n",
      "       'H_1': (2, 3, 4), 'H_2': (7, 8, 9), \n",
      "       'site_1': (0, 1, 2, 3, 4), \n",
      "       'site_2': (5, 6, 7, 8, 9)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_1 & signal_2) # Merging two Signal Sets --> keeping the synchronous part of signal_1 and signal_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting channels and runs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can define a mask in order to locate 512 Hz sampling data only"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True,  True, False])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask = signal_2.sampling_rates == 512 \n",
    "mask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We select the 512 Hz run by using the appropriate mask\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 2 runs\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:00  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.select_runs(mask))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can select all the other data not contained in the mask by using `~mask`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 1 run\n",
      "tags: {'Ex_2': (0,), 'Ey_2': (1,), 'Hx_2': (2,), \n",
      "       'Hy_2': (3,), 'Hz_2': (4,), 'E_2': (0, 1), \n",
      "       'H_2': (2, 3, 4), 'site_2': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(signal_2.select_runs(~mask))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Store, search & combine: the inventory\n",
    "\n",
    "[Inventory](../api/razorback.signalset.rst#razorback.signalset.Inventory)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create an Inventory Object from `all_signals`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "inventory = rb.Inventory(all_signals) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can print all the tags of the Inventory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'E_1',\n",
       " 'E_2',\n",
       " 'E_3',\n",
       " 'E_4',\n",
       " 'E_5',\n",
       " 'Ex_1',\n",
       " 'Ex_2',\n",
       " 'Ex_3',\n",
       " 'Ex_4',\n",
       " 'Ex_5',\n",
       " 'Ey_1',\n",
       " 'Ey_2',\n",
       " 'Ey_3',\n",
       " 'Ey_4',\n",
       " 'Ey_5',\n",
       " 'H_1',\n",
       " 'H_2',\n",
       " 'H_3',\n",
       " 'H_4',\n",
       " 'H_5',\n",
       " 'Hx_1',\n",
       " 'Hx_2',\n",
       " 'Hx_3',\n",
       " 'Hx_4',\n",
       " 'Hx_5',\n",
       " 'Hy_1',\n",
       " 'Hy_2',\n",
       " 'Hy_3',\n",
       " 'Hy_4',\n",
       " 'Hy_5',\n",
       " 'Hz_1',\n",
       " 'Hz_2',\n",
       " 'Hz_3',\n",
       " 'Hz_4',\n",
       " 'Hz_5',\n",
       " 'site_1',\n",
       " 'site_2',\n",
       " 'site_3',\n",
       " 'site_4',\n",
       " 'site_5'}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inventory.tags"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can select sub-inventory/SignalSet using the tag of the MT site `site_3` and show its tags:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inventory([SignalSet({'Ex_3': (0,), 'Ey_3': (1,), 'Hx_3': (2,), 'Hy_3': (3,), 'Hz_3': (4,), 'E_3': (0, 1), 'H_3': (2, 3, 4), 'site_3': (0, 1, 2, 3, 4)}, SyncSignal([5x148481], sampling_rate=5.1e+02, start=10, calibrations=[...])), SignalSet({'Ex_3': (0,), 'Ey_3': (1,), 'Hx_3': (2,), 'Hy_3': (3,), 'Hz_3': (4,), 'E_3': (0, 1), 'H_3': (2, 3, 4), 'site_3': (0, 1, 2, 3, 4)}, SyncSignal([5x153601], sampling_rate=1e+03, start=3.5e+02, calibrations=[...]))])\n",
      "\n",
      "{'Ex_3', 'Ey_3', 'Hx_3', 'H_3', 'Hy_3', 'site_3', 'Hz_3', 'E_3'}\n"
     ]
    }
   ],
   "source": [
    "sub_inventory = inventory.select_channels('site_3')\n",
    "print(sub_inventory)\n",
    "print()\n",
    "print(sub_inventory.tags)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pack instruction creates the largest (longest) signal set that contains `sub_inventory.tags`  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 5 channels, 2 runs\n",
      "tags: {'Ex_3': (0,), 'Ey_3': (1,), 'Hx_3': (2,), \n",
      "       'Hy_3': (3,), 'Hz_3': (4,), 'E_3': (0, 1), \n",
      "       'H_3': (2, 3, 4), 'site_3': (0, 1, 2, 3, 4)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:10  1970-01-01 00:05:00\n",
      "      1024  1970-01-01 00:05:50  1970-01-01 00:08:20\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(sub_inventory.pack()) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, first we create a sub-inventory using tags of the MT sites `site_2` and `site_3`, then we create a SignalSet object out of it (using `pack()`): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 10 channels, 3 runs\n",
      "tags: {'Ex_2': (0,), 'Ex_3': (5,), 'Ey_2': (1,), \n",
      "       'Ey_3': (6,), 'Hx_2': (2,), 'Hx_3': (7,), \n",
      "       'Hy_2': (3,), 'Hy_3': (8,), 'Hz_2': (4,), \n",
      "       'Hz_3': (9,), 'E_2': (0, 1), 'E_3': (5, 6), \n",
      "       'H_2': (2, 3, 4), 'H_3': (7, 8, 9), \n",
      "       'site_2': (0, 1, 2, 3, 4), \n",
      "       'site_3': (5, 6, 7, 8, 9)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:10  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "      1024  1970-01-01 00:06:40  1970-01-01 00:07:30\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(inventory.select_channels('site_2', 'site_3').pack())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same operation as above with  `site_3` and `site_5`. Please note that both sites have no common synchronous recordings at 512Hz. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 10 channels, 1 run\n",
      "tags: {'Ex_3': (0,), 'Ex_5': (5,), 'Ey_3': (1,), \n",
      "       'Ey_5': (6,), 'Hx_3': (2,), 'Hx_5': (7,), \n",
      "       'Hy_3': (3,), 'Hy_5': (8,), 'Hz_3': (4,), \n",
      "       'Hz_5': (9,), 'E_3': (0, 1), 'E_5': (5, 6), \n",
      "       'H_3': (2, 3, 4), 'H_5': (7, 8, 9), \n",
      "       'site_3': (0, 1, 2, 3, 4), \n",
      "       'site_5': (5, 6, 7, 8, 9)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "      1024  1970-01-01 00:05:50  1970-01-01 00:08:20\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(inventory.select_channels('site_3', 'site_5').pack()) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same operation as above with  `site_2` and `site_4`. Please note that both sites have ONLY common synchronous recordings at 512Hz. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 10 channels, 2 runs\n",
      "tags: {'Ex_2': (0,), 'Ex_4': (5,), 'Ey_2': (1,), \n",
      "       'Ey_4': (6,), 'Hx_2': (2,), 'Hx_4': (7,), \n",
      "       'Hy_2': (3,), 'Hy_4': (8,), 'Hz_2': (4,), \n",
      "       'Hz_4': (9,), 'E_2': (0, 1), 'E_4': (5, 6), \n",
      "       'H_2': (2, 3, 4), 'H_4': (7, 8, 9), \n",
      "       'site_2': (0, 1, 2, 3, 4), \n",
      "       'site_4': (5, 6, 7, 8, 9)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:05  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(inventory.select_channels('site_2', 'site_4').pack())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When there is no synchronousness between the different `SyncSignals` channels-, `pack()` instruction returns `None`. \n",
    "Consequently, the MT user needs to check the synchronoussness of the different channels.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(inventory.pack())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same operation as above but a filter is applied in order to select electric fields only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SignalSet: 4 channels, 2 runs\n",
      "tags: {'Ex_2': (0,), 'Ex_4': (2,), 'Ey_2': (1,), \n",
      "       'Ey_4': (3,), 'E_2': (0, 1), 'E_4': (2, 3)}\n",
      "----------  -------------------  -------------------\n",
      "  sampling                start                 stop\n",
      "       512  1970-01-01 00:00:05  1970-01-01 00:01:40\n",
      "       512  1970-01-01 00:02:00  1970-01-01 00:03:40\n",
      "----------  -------------------  -------------------\n"
     ]
    }
   ],
   "source": [
    "print(inventory.select_channels('site_2', 'site_4').filter('E_*').pack()) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}