{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Conversion to time series\n",
    "\n",
    "## c3s_sm reshuffle\n",
    "\n",
    "By using the readers described in [the previous chapter](#With-c3s_sm.interface), the program `c3s_sm reshuffle` converts images in a directory into a time series format for performant reading (https://github.com/TUW-GEO/pynetcf).\n",
    "\n",
    "This package provides a command line tool `c3s_sm reshuffle` for this purpose. e.g the following\n",
    "shell command will take all images for the year 2020 in the give image directory and store soil moisture data in the bounding box from 5-15 °E and 47-55 °N as time series in the chosen output path.\n",
    "\n",
    "```\n",
    "c3s_sm reshuffle /path/to/images /output/path -s 2019-10-01 -e 2020-01-31 -p sm --bbox 5 47 15 55\n",
    "```\n",
    "\n",
    "The same cane be done via the python API"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "42c31dcfd1489a4d"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "%%capture\n",
    "from c3s_sm.reshuffle import img2ts\n",
    "from tempfile import mkdtemp\n",
    "\n",
    "in_path = '../../tests/c3s_sm-test-data/img/TCDR/061_monthlyImages/combined'\n",
    "out_path = mkdtemp()\n",
    "\n",
    "img2ts(in_path, out_path, startdate='2016-04-01', enddate='2016-05-01', bbox=[5, 47, 15, 55])\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-08-29T14:04:10.892639Z",
     "start_time": "2024-08-29T14:04:09.888532Z"
    }
   },
   "id": "7150a4ab7e1d266e",
   "execution_count": 1
  },
  {
   "cell_type": "markdown",
   "source": [
    "This will generate data cubes of 5x5 degrees (compare figure below) that contain up to \n",
    "400 time series each.\n",
    "\n",
    "<img src=\"5x5_cell_partitioning.png\" width=500>\n",
    "\n",
    "Afterwards, we can use the time series reader class and read the data at one of the converted locations (if we pass a coordinate pair, the nearest grid cell is used). Note that for this example we only used 2 (monthly) test data images."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "4eb928c26ab2b86a"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance to nearest grid cell at Lat: 49 °N, Lon: 9 °E:  16634.19 metres\n"
     ]
    },
    {
     "data": {
      "text/plain": "            nobs  sensor  freqbandID        sm\n2016-04-01    30     512           2  0.298179\n2016-05-01    31     768           2  0.267991",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>nobs</th>\n      <th>sensor</th>\n      <th>freqbandID</th>\n      <th>sm</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2016-04-01</th>\n      <td>30</td>\n      <td>512</td>\n      <td>2</td>\n      <td>0.298179</td>\n    </tr>\n    <tr>\n      <th>2016-05-01</th>\n      <td>31</td>\n      <td>768</td>\n      <td>2</td>\n      <td>0.267991</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from c3s_sm.interface import C3STs\n",
    "reader = C3STs(out_path, ioclass_kws={'read_bulk': True})\n",
    "gpi, dist = reader.grid.find_nearest_gpi(9, 49)\n",
    "print(\"Distance to nearest grid cell at Lat: 49 °N, Lon: 9 °E: \", f\"{dist:.02f}\", 'metres')\n",
    "ts = reader.read(gpi)   # or reader.read(9, 49)\n",
    "reader.close()\n",
    "ts\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-08-29T14:04:10.923498Z",
     "start_time": "2024-08-29T14:04:10.893564Z"
    }
   },
   "id": "765720e58ae05810",
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "source": [
    "## c3s_sm update_ts\n",
    "\n",
    "In line with the program `c3s_sm update_img` there is also `c3s_sm update_ts`\n",
    "is used to extend a time series dataset with the newest image data (e.g.\n",
    "as part of a recurring job).\n",
    "\n",
    "```\n",
    "c3s_sm update_ts /image/path /time/series/path\n",
    "``` \n",
    "\n",
    "The same can be done in python"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7afceabc5885fe35"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "from c3s_sm.reshuffle import extend_ts\n",
    "\n",
    "extend_ts(in_path, ts_path=out_path)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-08-29T14:04:11.223592Z",
     "start_time": "2024-08-29T14:04:10.924243Z"
    }
   },
   "id": "d002bfb3a2f714b2",
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "source": [
    "If we read the same time series again now, we see that data from an additional\n",
    "image was appended."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d8d7beed8fa8f275"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "            nobs  sensor  freqbandID        sm\n2016-04-01    30     512           2  0.298179\n2016-05-01    31     768           2  0.267991\n2016-06-01    30     768           2  0.292918",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>nobs</th>\n      <th>sensor</th>\n      <th>freqbandID</th>\n      <th>sm</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2016-04-01</th>\n      <td>30</td>\n      <td>512</td>\n      <td>2</td>\n      <td>0.298179</td>\n    </tr>\n    <tr>\n      <th>2016-05-01</th>\n      <td>31</td>\n      <td>768</td>\n      <td>2</td>\n      <td>0.267991</td>\n    </tr>\n    <tr>\n      <th>2016-06-01</th>\n      <td>30</td>\n      <td>768</td>\n      <td>2</td>\n      <td>0.292918</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "reader = C3STs(out_path, ioclass_kws={'read_bulk': True})\n",
    "ts = reader.read(gpi)   # or reader.read(9, 49)\n",
    "ts"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-08-29T14:04:11.240366Z",
     "start_time": "2024-08-29T14:04:11.224298Z"
    }
   },
   "id": "bd12078a4047e1c8",
   "execution_count": 4
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}