{ "cells": [ { "cell_type": "markdown", "id": "b1eefe41", "metadata": {}, "source": [ "# 4. Conditional Control of `datetime` Index" ] }, { "cell_type": "markdown", "id": "ae5b58b4", "metadata": {}, "source": [ "## Conditional Control of `datetime` Index in `xarray`\n", "\n", "In Unit 3, we demonstrated how to use the `sel` method with `slice` to select data with a continuous temporal or spatial range. However, sometimes we need to select non-continuous time periods, such as certain months over several years. In these cases, it is not useful to select with `slice`. Therefore, we can use conditional control arguments to select data that meet the requirements we specify. Specifically, the time coordinate in xarray is a datetime object, which includes datetime attributes such as year, month, day, and so on. We can use these attributes to select the dates we like. \n", "\n", "**Example 1: Select only the JAS season data.** " ] }, { "cell_type": "code", "execution_count": 1, "id": "0721be78", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray 'olr' (time: 2208, lat: 90, lon: 360)> Size: 286MB\n",
       "[71539200 values with dtype=float32]\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 18kB 1998-07-01 1998-07-02 ... 2021-09-30\n",
       "  * lon      (lon) float32 1kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5\n",
       "  * lat      (lat) float32 360B -44.5 -43.5 -42.5 -41.5 ... 41.5 42.5 43.5 44.5\n",
       "Attributes:\n",
       "    standard_name:  toa_outgoing_longwave_flux\n",
       "    long_name:      NOAA Climate Data Record of Daily Mean Upward Longwave Fl...\n",
       "    units:          W m-2\n",
       "    cell_methods:   time: mean area: mean
" ], "text/plain": [ " Size: 286MB\n", "[71539200 values with dtype=float32]\n", "Coordinates:\n", " * time (time) datetime64[ns] 18kB 1998-07-01 1998-07-02 ... 2021-09-30\n", " * lon (lon) float32 1kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5\n", " * lat (lat) float32 360B -44.5 -43.5 -42.5 -41.5 ... 41.5 42.5 43.5 44.5\n", "Attributes:\n", " standard_name: toa_outgoing_longwave_flux\n", " long_name: NOAA Climate Data Record of Daily Mean Upward Longwave Fl...\n", " units: W m-2\n", " cell_methods: time: mean area: mean" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import xarray as xr \n", "\n", "olr_da = xr.open_dataset(\"data/olr.nc\").olr\n", "olr_jas = olr_da.sel(time=(olr_da.time.dt.month.isin([7,8,9]))) \n", "olr_jas" ] }, { "cell_type": "markdown", "id": "04acd508", "metadata": {}, "source": [ "For `time=(olr_da.time.dt.month.isin([7, 8, 9]))`, xarray will check if the month of each timestep falls in either the 7th, 8th, or 9th month (i.e., July, August, or September). If so, the timestep will be marked `True`. Otherwise, it will be marked `False`. Finally, only the data points marked `True` will be preserved." ] }, { "cell_type": "markdown", "id": "38da4c51", "metadata": {}, "source": [ "**Example 2: Remove Leap Days**\n", "\n", "Similar to Example 1, we can use *reverse selection* to remove the leap days. This means selecting all dates that are not February 29th.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "b01a127f", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray 'olr' (time: 8760, lat: 90, lon: 360)> Size: 1GB\n",
       "[283824000 values with dtype=float32]\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 70kB 1998-01-01 1998-01-02 ... 2021-12-31\n",
       "  * lon      (lon) float32 1kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5\n",
       "  * lat      (lat) float32 360B -44.5 -43.5 -42.5 -41.5 ... 41.5 42.5 43.5 44.5\n",
       "Attributes:\n",
       "    standard_name:  toa_outgoing_longwave_flux\n",
       "    long_name:      NOAA Climate Data Record of Daily Mean Upward Longwave Fl...\n",
       "    units:          W m-2\n",
       "    cell_methods:   time: mean area: mean
" ], "text/plain": [ " Size: 1GB\n", "[283824000 values with dtype=float32]\n", "Coordinates:\n", " * time (time) datetime64[ns] 70kB 1998-01-01 1998-01-02 ... 2021-12-31\n", " * lon (lon) float32 1kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5\n", " * lat (lat) float32 360B -44.5 -43.5 -42.5 -41.5 ... 41.5 42.5 43.5 44.5\n", "Attributes:\n", " standard_name: toa_outgoing_longwave_flux\n", " long_name: NOAA Climate Data Record of Daily Mean Upward Longwave Fl...\n", " units: W m-2\n", " cell_methods: time: mean area: mean" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "olr_noleap = olr_da.sel(time=~((olr_da.time.dt.month == 2) & (olr_da.time.dt.day == 29))) # ~(): reversed selection\n", " # not selecting 2/29\n", "olr_noleap" ] }, { "cell_type": "markdown", "id": "4d959a2f", "metadata": {}, "source": [ "## DatetimeIndex and Its Applications\n", "\n", "Using `pandas`, we can easily create a datetime object. The `to_datetime` method can convert string with datetime format to a datetime object. " ] }, { "cell_type": "code", "execution_count": 3, "id": "44ef1bbc", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatetimeIndex(['2000-01-01', '2000-02-02'], dtype='datetime64[ns]', freq=None)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "pd.to_datetime([\"2000-01-01\", \"2000-02-02\"])" ] }, { "cell_type": "markdown", "id": "91fca480", "metadata": {}, "source": [ "We can also specify the start time and the length to create the time series. " ] }, { "cell_type": "code", "execution_count": 4, "id": "045d63c6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04',\n", " '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-08',\n", " '2000-01-09', '2000-01-10',\n", " ...\n", " '2000-12-21', '2000-12-22', '2000-12-23', '2000-12-24',\n", " '2000-12-25', '2000-12-26', '2000-12-27', '2000-12-28',\n", " '2000-12-29', '2000-12-30'],\n", " dtype='datetime64[ns]', length=365, freq='D')" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ts = pd.date_range(\"2000-01-01\", periods=365)\n", "ts" ] }, { "cell_type": "markdown", "id": "431ce71f", "metadata": {}, "source": [ "Or specify the start and end time, and sampling frequency. " ] }, { "cell_type": "code", "execution_count": 5, "id": "2384bbb5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04',\n", " '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-08',\n", " '2000-01-09', '2000-01-10',\n", " ...\n", " '2000-12-21', '2000-12-22', '2000-12-23', '2000-12-24',\n", " '2000-12-25', '2000-12-26', '2000-12-27', '2000-12-28',\n", " '2000-12-29', '2000-12-30'],\n", " dtype='datetime64[ns]', length=365, freq='D')" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ts = pd.date_range(start='2000-01-01',end='2000-12-30',freq='1D')\n", "ts" ] }, { "cell_type": "markdown", "id": "14056d83", "metadata": {}, "source": [ "To convert the datetime with formatted strings, we can use `strftime` method. For example, we format the datetime to 'Jan 01 00' here: " ] }, { "cell_type": "code", "execution_count": 6, "id": "81b68128", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['Jan 01 00', 'Jan 02 00', 'Jan 03 00', 'Jan 04 00', 'Jan 05 00',\n", " 'Jan 06 00', 'Jan 07 00', 'Jan 08 00', 'Jan 09 00', 'Jan 10 00',\n", " ...\n", " 'Dec 21 00', 'Dec 22 00', 'Dec 23 00', 'Dec 24 00', 'Dec 25 00',\n", " 'Dec 26 00', 'Dec 27 00', 'Dec 28 00', 'Dec 29 00', 'Dec 30 00'],\n", " dtype='object', length=365)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ts.strftime(\"%b %d %y\")" ] }, { "cell_type": "markdown", "id": "d247478a", "metadata": {}, "source": [ "Note that this string index is no longer a datetime object. The **formatter** `%b` means to format months as abbreviated names, and `%y` means year without century as a zero-padded decimal number. Detailed usages of the formatters can be found in [Datetime: `strftime`-`strptime` Behavior](https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior).\n", "\n", "Similarly, we can format the time coordinate of a DataArray into a string format:" ] }, { "cell_type": "code", "execution_count": 7, "id": "f9d46c71", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray 'strftime' (time: 8760)> Size: 70kB\n",
       "array(['Jan 01 98', 'Jan 02 98', 'Jan 03 98', ..., 'Dec 29 21',\n",
       "       'Dec 30 21', 'Dec 31 21'], dtype=object)\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 70kB 1998-01-01 1998-01-02 ... 2021-12-31
" ], "text/plain": [ " Size: 70kB\n", "array(['Jan 01 98', 'Jan 02 98', 'Jan 03 98', ..., 'Dec 29 21',\n", " 'Dec 30 21', 'Dec 31 21'], dtype=object)\n", "Coordinates:\n", " * time (time) datetime64[ns] 70kB 1998-01-01 1998-01-02 ... 2021-12-31" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "olr_da.time.dt.strftime(\"%b %d %y\")" ] }, { "cell_type": "markdown", "id": "e9d0e720", "metadata": {}, "source": [ "Therefore, the **DatetimeAccessor** `xarray.DataArray.time.dt` is equivalent to a `pandas.DatetimeIndex`.\n", "\n", "It is important to learn the `strftime` method because it will be applicable to formatting the time labels on time series plots or Hovmöller diagrams." ] }, { "cell_type": "markdown", "id": "fa7f2b28", "metadata": {}, "source": [ "## `datetime` and `timedelta`\n", "\n", "Datetime Accessor and `pandas.DatetimeIndex` actually belong to datetime objects.\n", "\n", "> `datetime.datetime`: A combination of a date and a time. Attributes: year, month, day, hour, minute, second, microsecond, and tzinfo. [(`datetime` offical website)](https://docs.python.org/3/library/datetime.html)\n", "\n", "We can also perform arithmetic calculations on datetime objects. For example, we can use the combination of `datetime.datetime` and `datetime.timedelta` to obtain a certain date.\n", "\n", "> A timedelta object represents a duration, the difference between two dates or times. [(`datetime` offical website)](https://docs.python.org/3/library/datetime.html)\n", "\n", "The following are some arithmetic rules for `datetime.datetime` and `datetime.timedelta`:\n", "\n", "```\n", "datetime2 = datetime1 + timedelta \n", "datetime2 = datetime1 - timedelta\n", "timedelta = datetime1 - datetime2\n", "datetime1 < datetime2 \n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.14" } }, "nbformat": 4, "nbformat_minor": 5 }