diff --git a/config/redirects b/config/redirects index 0794caeb3df..5362eae38aa 100644 --- a/config/redirects +++ b/config/redirects @@ -1791,6 +1791,13 @@ raw: /manual/core/wildcard -> ${base}/manual/core/index-wildcard/ [v4.4-*]: /${version}/genindex -> ${base}/${version}/ +# +# Redirects for new 5.1 pages +# + +[v3.6-v5.0]: /${version}/reference/operator/aggregation/tsIncrement/ -> ${base}/${version}/reference/operator/aggregation/ +[v3.6-v5.0]: /${version}/reference/operator/aggregation/tsSecond/ -> ${base}/${version}/reference/operator/aggregation/ + # # Redirects for new 5.0 pages # diff --git a/source/changeStreams.txt b/source/changeStreams.txt index f55a0a657db..a89c048c9cd 100644 --- a/source/changeStreams.txt +++ b/source/changeStreams.txt @@ -27,6 +27,10 @@ immediately react to them. Because change streams use the aggregation framework, applications can also filter for specific changes or transform the notifications at will. +.. |change-streams| replace:: change streams + +.. include:: /includes/change-streams-optimization.rst + Availability ------------ diff --git a/source/includes/change-streams-optimization.rst b/source/includes/change-streams-optimization.rst new file mode 100644 index 00000000000..801b2eef22b --- /dev/null +++ b/source/includes/change-streams-optimization.rst @@ -0,0 +1,3 @@ +Starting in MongoDB 5.1, |change-streams| are optimized, providing more +efficient resource utilization and faster execution of some aggregation +pipeline stages. \ No newline at end of file diff --git a/source/includes/extracts-agg-operators.yaml b/source/includes/extracts-agg-operators.yaml index 895fa310d4d..904f5fcbdef 100644 --- a/source/includes/extracts-agg-operators.yaml +++ b/source/includes/extracts-agg-operators.yaml @@ -805,6 +805,32 @@ content: | - Access available per-document metadata related to the aggregation operation. +--- +ref: agg-operators-timestamp +content: | + + Timestamp expression operators return values from a :ref:`timestamp + `. + + .. list-table:: + :header-rows: 1 + :widths: 20 80 + + * - Name + - Description + + * - :expression:`$tsIncrement` + + - .. include:: /includes/tsIncrement-introduction.rst + + .. versionadded:: 5.1 + + * - :expression:`$tsSecond` + + - .. include:: /includes/tsSecond-introduction.rst + + .. versionadded:: 5.1 + --- ref: agg-operators-variable-project content: | diff --git a/source/includes/stockSales-example-collection-create.rst b/source/includes/stockSales-example-collection-create.rst new file mode 100644 index 00000000000..5d0412c6170 --- /dev/null +++ b/source/includes/stockSales-example-collection-create.rst @@ -0,0 +1,9 @@ +.. code-block:: javascript + + db.stockSales.insertMany( [ + { _id: 0, symbol: "ABC", saleTimestamp: Timestamp(1622731060, 1) }, + { _id: 1, symbol: "ABC", saleTimestamp: Timestamp(1622731060, 2) }, + { _id: 2, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 1) }, + { _id: 3, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 2) }, + { _id: 4, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 3) } + ] ) diff --git a/source/includes/stockSales-example-collection.rst b/source/includes/stockSales-example-collection.rst new file mode 100644 index 00000000000..a70d0f70256 --- /dev/null +++ b/source/includes/stockSales-example-collection.rst @@ -0,0 +1,13 @@ +Create a ``stockSales`` collection that contains company stock financial +market sales: + +.. include:: /includes/stockSales-example-collection-create.rst + +In the :ref:`timestamp ` constructor, the: + +- First value is the number of seconds after the :wikipedia:`Unix epoch + `. + +- Second value is the incrementing ordinal. When multiple events happen + within the same second, the incrementing ordinal uniquely identifies + each event. \ No newline at end of file diff --git a/source/includes/tsIncrement-introduction.rst b/source/includes/tsIncrement-introduction.rst new file mode 100644 index 00000000000..72074037fef --- /dev/null +++ b/source/includes/tsIncrement-introduction.rst @@ -0,0 +1,2 @@ +Returns the incrementing ordinal from a :ref:`timestamp +` as a :bsontype:`long `. \ No newline at end of file diff --git a/source/includes/tsSecond-introduction.rst b/source/includes/tsSecond-introduction.rst new file mode 100644 index 00000000000..cae80161291 --- /dev/null +++ b/source/includes/tsSecond-introduction.rst @@ -0,0 +1,2 @@ +Returns the seconds from a :ref:`timestamp +` as a :bsontype:`long `. \ No newline at end of file diff --git a/source/reference/operator/aggregation.txt b/source/reference/operator/aggregation.txt index d2d544bf6b6..5fc7dc61949 100644 --- a/source/reference/operator/aggregation.txt +++ b/source/reference/operator/aggregation.txt @@ -102,6 +102,11 @@ Text Expression Operator .. include:: /includes/extracts/agg-operators-text.rst +Timestamp Expression Operators +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/extracts/agg-operators-timestamp.rst + Trigonometry Expression Operators ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1100,6 +1105,20 @@ Alphabetical Listing of Expression Operators decimal place. + * - :expression:`$tsIncrement` + + - .. include:: /includes/tsIncrement-introduction.rst + + .. versionadded:: 5.1 + + + * - :expression:`$tsSecond` + + - .. include:: /includes/tsSecond-introduction.rst + + .. versionadded:: 5.1 + + * - :expression:`$type` - Return the BSON data type of the field. @@ -1280,6 +1299,8 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/toString /reference/operator/aggregation/toLower /reference/operator/aggregation/toUpper + /reference/operator/aggregation/tsIncrement + /reference/operator/aggregation/tsSecond /reference/operator/aggregation/trim /reference/operator/aggregation/trunc /reference/operator/aggregation/type diff --git a/source/reference/operator/aggregation/tsIncrement.txt b/source/reference/operator/aggregation/tsIncrement.txt new file mode 100644 index 00000000000..6e05b4b0e10 --- /dev/null +++ b/source/reference/operator/aggregation/tsIncrement.txt @@ -0,0 +1,188 @@ +========================== +$tsIncrement (aggregation) +========================== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. expression:: $tsIncrement + +.. versionadded:: 5.1 + +.. include:: /includes/tsIncrement-introduction.rst + +When multiple events happen within the same second, the incrementing +ordinal uniquely identifies each event. + +:expression:`$tsIncrement` syntax: + +.. code-block:: none + :copyable: false + + { $tsIncrement: } + +The :ref:`expression ` must resolve to a +:ref:`timestamp `. + +.. seealso:: + + - :ref:`aggregation-expressions` + - :ref:`bson-types` + - :expression:`$tsSecond` + +Behavior +-------- + +:expression:`$tsIncrement` returns: + +- ``Null`` if the input :ref:`expression ` + evaluates to ``null`` or refers to a field that is missing. + +- An error if the input :ref:`expression ` does + not evaluate to a :ref:`timestamp `. + +Examples +-------- + +Obtain the Incrementing Ordinal from a Timestamp Field +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/stockSales-example-collection.rst + +The following example uses :expression:`$tsIncrement` in a +:pipeline:`$project` stage to return the incrementing ordinal from the +stock sales ``saleTimestamp`` field: + +.. code-block:: javascript + + db.stockSales.aggregate( [ + { + $project: + { + _id: 0, saleTimestamp: 1, saleIncrement: { $tsIncrement: "$saleTimestamp" } + } + } + ] ) + +In the example, :pipeline:`$project` only includes the ``saleTimestamp`` +and ``saleIncrement`` fields as shown in the following output: + +.. code-block:: javascript + :copyable: false + + { + saleTimestamp: Timestamp({ t: 1622731060, i: 1 }), + saleIncrement: Long("1") + }, + { + saleTimestamp: Timestamp({ t: 1622731060, i: 2 }), + saleIncrement: Long("2") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 1 }), + saleIncrement: Long("1") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 2 }), + saleIncrement: Long("2") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 3 }), + saleIncrement: Long("3") + } + +Use ``$tsIncrement`` in a Change Stream Cursor to Monitor Collection Changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The example in this section uses :expression:`$tsIncrement` in a +:ref:`change stream cursor ` to return every other change +made to a collection in the same second of time. + +Create a :ref:`change stream cursor ` on a collection +named ``cakeSales`` that you will see later in this section: + +.. code-block:: javascript + + cakeSalesCursor = db.cakeSales.watch( [ + { + $match: { + $expr: { + $eq: [ + { $mod: [ { $tsIncrement: "$clusterTime" } , 2 ] }, + 0 + ] + } + } + } + ] ) + +In the example, the: + +- :method:`db.collection.watch()` method creates a :ref:`change stream + cursor ` for the ``cakeSales`` collection and stores + the cursor in ``cakeSalesCursor``. + +- :pipeline:`$match` stage filters the documents to those + returned by the :query:`$expr` operator. + +- :query:`$expr` operator: + + - Applies :expression:`$mod` ``2`` to the ``$clusterTime`` variable's + incrementing ordinal returned by :expression:`$tsIncrement`. + + ``$clusterTime`` is the timestamp from the :ref:`oplog + ` entry when the ``cakeSales`` collection is + modified. See :ref:`Command Response `. + + - Compares the returned value from :expression:`$mod` to ``0`` using + :expression:`$eq`. + +.. include:: /includes/cakeSales-example-collection.rst + +To monitor the ``cakeSales`` collection changes, use +``cakeSalesCursor``. For example, to obtain the next document from +``cakeSalesCursor``, use the :method:`~cursor.next()` method: + +.. code-block:: javascript + + cakeSalesCursor.next() + +Depending on the second when the documents were added to ``cakeSales``, +the output from ``cakeSalesCursor.next()`` varies. For example, the +document additions might span more than one second. + +The following ``cakeSalesCursor.next()`` example output shows the +``insert`` details for the first document added to the ``cakeSales`` +collection. Notice the incrementing ordinal ``i`` is ``2`` in the +``clusterTime`` field. + +.. code-block:: javascript + :copyable: false + + _id: { + _data: '82613A4F25000000022B022C0100296E5A100454C5BFAF538C47AB950614F43889BE00461E5F696400290004' + }, + operationType: 'insert', + clusterTime: Timestamp({ t: 1631211301, i: 2 }), + fullDocument: { + _id: 0, + type: 'chocolate', + orderDate: ISODate("2020-05-18T14:10:30.000Z"), + state: 'CA', + price: 13, + quantity: 120 + }, + ns: { db: 'test', coll: 'cakeSales' }, + documentKey: { _id: 0 } + +Running ``cakeSalesCursor.next()`` again returns the ``cakeSales`` +document for which the ``clusterTime`` incrementing ordinal ``i`` is +``4``, omitting the document where ``i`` is ``3``. \ No newline at end of file diff --git a/source/reference/operator/aggregation/tsSecond.txt b/source/reference/operator/aggregation/tsSecond.txt new file mode 100644 index 00000000000..b6c831c1be6 --- /dev/null +++ b/source/reference/operator/aggregation/tsSecond.txt @@ -0,0 +1,167 @@ +======================= +$tsSecond (aggregation) +======================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. expression:: $tsSecond + +.. versionadded:: 5.1 + +.. include:: /includes/tsSecond-introduction.rst + +:expression:`$tsSecond` syntax: + +.. code-block:: none + :copyable: false + + { $tsSecond: } + +The :ref:`expression ` must resolve to a +:ref:`timestamp `. + +.. seealso:: + + - :ref:`aggregation-expressions` + - :ref:`bson-types` + - :expression:`$tsIncrement` + +Behavior +-------- + +:expression:`$tsSecond` returns: + +- ``Null`` if the input :ref:`expression ` + evaluates to ``null`` or refers to a field that is missing. + +- An error if the input :ref:`expression ` does + not evaluate to a :ref:`timestamp `. + +Examples +-------- + +Obtain the Number of Seconds from a Timestamp Field +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/stockSales-example-collection.rst + +The following example uses :expression:`$tsSecond` in a +:pipeline:`$project` pipeline stage to return the seconds from the +stock sales ``saleTimestamp`` field: + +.. code-block:: javascript + + db.stockSales.aggregate( [ + { + $project: + { + _id: 0, saleTimestamp: 1, saleSeconds: { $tsSecond: "$saleTimestamp" } + } + } + ] ) + +Example output: + +.. code-block:: javascript + :copyable: false + + { + saleTimestamp: Timestamp({ t: 1622731060, i: 1 }), + saleSeconds: Long("1622731060") + }, + { + saleTimestamp: Timestamp({ t: 1622731060, i: 2 }), + saleSeconds: Long("1622731060") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 1 }), + saleSeconds: Long("1714124193") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 2 }), + saleSeconds: Long("1714124193") + }, + { + saleTimestamp: Timestamp({ t: 1714124193, i: 3 }), + saleSeconds: Long("1714124193") + } + +Use ``$tsSecond`` in a Change Stream Cursor to Monitor Collection Changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The example in this section uses :expression:`$tsSecond` in a +:ref:`change stream cursor ` to monitor changes to a +collection. + +Create a :ref:`change stream cursor ` on a collection +named ``cakeSales`` that you will see later in this section: + +.. code-block:: javascript + + cakeSalesCursor = db.cakeSales.watch( [ + { + $addFields: { + clusterTimeSeconds: { $tsSecond: "$clusterTime" } + } + } + ] ) + +In the example, the: + +- :method:`db.collection.watch()` method creates a :ref:`change stream + cursor ` for the ``cakeSales`` collection and stores + the cursor in ``cakeSalesCursor``. + +- :pipeline:`$addFields` stage adds a field named ``clusterTimeSeconds`` + to ``cakeSalesCursor``. + + - ``$clusterTime`` is the timestamp from the :ref:`oplog + ` entry for the ``cakeSales`` collection change. + See :ref:`Command Response `. + + - :expression:`$tsSecond` returns the seconds from ``$clusterTime``, + which is stored in ``clusterTimeSeconds``. + +.. include:: /includes/cakeSales-example-collection.rst + +To monitor the ``cakeSales`` collection changes, use +``cakeSalesCursor``. For example, to obtain the next document from +``cakeSalesCursor``, use the :method:`~cursor.next()` method: + +.. code-block:: javascript + + cakeSalesCursor.next() + +The following example output shows the ``insert`` details for the first +document added to the ``cakeSales`` collection. The +``clusterTimeSeconds`` field contains the seconds from the +``clusterTime`` field. + +.. code-block:: javascript + :copyable: false + + _id: { + _data: '82613A4A51000000032B022C0100296E5A100495189B4131584C56AC8BA9D540799F23461E5F696400290004' + }, + operationType: 'insert', + clusterTime: Timestamp({ t: 1631210065, i: 3 }), + fullDocument: { + _id: 0, + type: 'chocolate', + orderDate: ISODate("2020-05-18T14:10:30.000Z"), + state: 'CA', + price: 13, + quantity: 120 + }, + ns: { db: 'test', coll: 'cakeSales' }, + documentKey: { _id: 0 }, + clusterTimeSeconds: 1631210065 diff --git a/source/release-notes/5.1.txt b/source/release-notes/5.1.txt index 25d71b094bc..8a04e0e9675 100644 --- a/source/release-notes/5.1.txt +++ b/source/release-notes/5.1.txt @@ -15,6 +15,41 @@ General .. include:: /includes/in-dev.rst +Aggregation +----------- + +.. _5.1-rel-notes-new-agg-operators: + +New Aggregation Operators +~~~~~~~~~~~~~~~~~~~~~~~~~ + +MongoDB 5.1 introduces the following aggregation operators: + +.. list-table:: + :header-rows: 1 + :widths: 20 80 + + * - Operator + - Description + + * - :expression:`$tsSecond` + - .. include:: /includes/tsSecond-introduction.rst + + * - :expression:`$tsIncrement` + - .. include:: /includes/tsIncrement-introduction.rst + +.. _5.1-rel-notes-change-streams: + +Change Streams +-------------- + +Optimization +~~~~~~~~~~~~ + +.. |change-streams| replace:: :ref:`change streams ` + +.. include:: /includes/change-streams-optimization.rst + Indexes ------- @@ -72,4 +107,3 @@ of the related projects. :hidden: /release-notes/5.1-compatibility -