Skip to content

Commit f5da63f

Browse files
jason-price-mongodbjeff-allen-mongo
jason-price-mongodb
authored and
jeff-allen-mongo
committed
DOCSP-15253 changeStream optimization
DOCSP-15253 changeStream optimization
1 parent 959a69f commit f5da63f

12 files changed

+477
-1
lines changed

‎config/redirects

+7
Original file line numberDiff line numberDiff line change
@@ -1791,6 +1791,13 @@ raw: /manual/core/wildcard -> ${base}/manual/core/index-wildcard/
17911791

17921792
[v4.4-*]: /${version}/genindex -> ${base}/${version}/
17931793

1794+
#
1795+
# Redirects for new 5.1 pages
1796+
#
1797+
1798+
[v3.6-v5.0]: /${version}/reference/operator/aggregation/tsIncrement/ -> ${base}/${version}/reference/operator/aggregation/
1799+
[v3.6-v5.0]: /${version}/reference/operator/aggregation/tsSecond/ -> ${base}/${version}/reference/operator/aggregation/
1800+
17941801
#
17951802
# Redirects for new 5.0 pages
17961803
#

‎source/changeStreams.txt

+4
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ immediately react to them. Because change streams use the aggregation
2727
framework, applications can also filter for specific changes or
2828
transform the notifications at will.
2929

30+
.. |change-streams| replace:: change streams
31+
32+
.. include:: /includes/change-streams-optimization.rst
33+
3034
Availability
3135
------------
3236

‎source/includes/change-streams-optimization.rst

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Starting in MongoDB 5.1, |change-streams| are optimized, providing more
2+
efficient resource utilization and faster execution of some aggregation
3+
pipeline stages.

‎source/includes/extracts-agg-operators.yaml

+26
Original file line numberDiff line numberDiff line change
@@ -805,6 +805,32 @@ content: |
805805
- Access available per-document metadata related to the
806806
aggregation operation.
807807
808+
---
809+
ref: agg-operators-timestamp
810+
content: |
811+
812+
Timestamp expression operators return values from a :ref:`timestamp
813+
<document-bson-type-timestamp>`.
814+
815+
.. list-table::
816+
:header-rows: 1
817+
:widths: 20 80
818+
819+
* - Name
820+
- Description
821+
822+
* - :expression:`$tsIncrement`
823+
824+
- .. include:: /includes/tsIncrement-introduction.rst
825+
826+
.. versionadded:: 5.1
827+
828+
* - :expression:`$tsSecond`
829+
830+
- .. include:: /includes/tsSecond-introduction.rst
831+
832+
.. versionadded:: 5.1
833+
808834
---
809835
ref: agg-operators-variable-project
810836
content: |

‎source/includes/stockSales-example-collection-create.rst

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
.. code-block:: javascript
2+
3+
db.stockSales.insertMany( [
4+
{ _id: 0, symbol: "ABC", saleTimestamp: Timestamp(1622731060, 1) },
5+
{ _id: 1, symbol: "ABC", saleTimestamp: Timestamp(1622731060, 2) },
6+
{ _id: 2, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 1) },
7+
{ _id: 3, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 2) },
8+
{ _id: 4, symbol: "DEF", saleTimestamp: Timestamp(1714124193, 3) }
9+
] )

‎source/includes/stockSales-example-collection.rst

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Create a ``stockSales`` collection that contains company stock financial
2+
market sales:
3+
4+
.. include:: /includes/stockSales-example-collection-create.rst
5+
6+
In the :ref:`timestamp <document-bson-type-timestamp>` constructor, the:
7+
8+
- First value is the number of seconds after the :wikipedia:`Unix epoch
9+
<Unix_time>`.
10+
11+
- Second value is the incrementing ordinal. When multiple events happen
12+
within the same second, the incrementing ordinal uniquely identifies
13+
each event.

‎source/includes/tsIncrement-introduction.rst

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Returns the incrementing ordinal from a :ref:`timestamp
2+
<document-bson-type-timestamp>` as a :bsontype:`long <data_numberlong>`.

‎source/includes/tsSecond-introduction.rst

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Returns the seconds from a :ref:`timestamp
2+
<document-bson-type-timestamp>` as a :bsontype:`long <data_numberlong>`.

‎source/reference/operator/aggregation.txt

+21
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,11 @@ Text Expression Operator
102102

103103
.. include:: /includes/extracts/agg-operators-text.rst
104104

105+
Timestamp Expression Operators
106+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
107+
108+
.. include:: /includes/extracts/agg-operators-timestamp.rst
109+
105110
Trigonometry Expression Operators
106111
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
107112

@@ -1100,6 +1105,20 @@ Alphabetical Listing of Expression Operators
11001105
decimal place.
11011106

11021107

1108+
* - :expression:`$tsIncrement`
1109+
1110+
- .. include:: /includes/tsIncrement-introduction.rst
1111+
1112+
.. versionadded:: 5.1
1113+
1114+
1115+
* - :expression:`$tsSecond`
1116+
1117+
- .. include:: /includes/tsSecond-introduction.rst
1118+
1119+
.. versionadded:: 5.1
1120+
1121+
11031122
* - :expression:`$type`
11041123

11051124
- Return the BSON data type of the field.
@@ -1280,6 +1299,8 @@ Alphabetical Listing of Expression Operators
12801299
/reference/operator/aggregation/toString
12811300
/reference/operator/aggregation/toLower
12821301
/reference/operator/aggregation/toUpper
1302+
/reference/operator/aggregation/tsIncrement
1303+
/reference/operator/aggregation/tsSecond
12831304
/reference/operator/aggregation/trim
12841305
/reference/operator/aggregation/trunc
12851306
/reference/operator/aggregation/type

‎source/reference/operator/aggregation/tsIncrement.txt

+188
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
==========================
2+
$tsIncrement (aggregation)
3+
==========================
4+
5+
.. default-domain:: mongodb
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
Definition
14+
----------
15+
16+
.. expression:: $tsIncrement
17+
18+
.. versionadded:: 5.1
19+
20+
.. include:: /includes/tsIncrement-introduction.rst
21+
22+
When multiple events happen within the same second, the incrementing
23+
ordinal uniquely identifies each event.
24+
25+
:expression:`$tsIncrement` syntax:
26+
27+
.. code-block:: none
28+
:copyable: false
29+
30+
{ $tsIncrement: <expression> }
31+
32+
The :ref:`expression <aggregation-expressions>` must resolve to a
33+
:ref:`timestamp <document-bson-type-timestamp>`.
34+
35+
.. seealso::
36+
37+
- :ref:`aggregation-expressions`
38+
- :ref:`bson-types`
39+
- :expression:`$tsSecond`
40+
41+
Behavior
42+
--------
43+
44+
:expression:`$tsIncrement` returns:
45+
46+
- ``Null`` if the input :ref:`expression <aggregation-expressions>`
47+
evaluates to ``null`` or refers to a field that is missing.
48+
49+
- An error if the input :ref:`expression <aggregation-expressions>` does
50+
not evaluate to a :ref:`timestamp <document-bson-type-timestamp>`.
51+
52+
Examples
53+
--------
54+
55+
Obtain the Incrementing Ordinal from a Timestamp Field
56+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
57+
58+
.. include:: /includes/stockSales-example-collection.rst
59+
60+
The following example uses :expression:`$tsIncrement` in a
61+
:pipeline:`$project` stage to return the incrementing ordinal from the
62+
stock sales ``saleTimestamp`` field:
63+
64+
.. code-block:: javascript
65+
66+
db.stockSales.aggregate( [
67+
{
68+
$project:
69+
{
70+
_id: 0, saleTimestamp: 1, saleIncrement: { $tsIncrement: "$saleTimestamp" }
71+
}
72+
}
73+
] )
74+
75+
In the example, :pipeline:`$project` only includes the ``saleTimestamp``
76+
and ``saleIncrement`` fields as shown in the following output:
77+
78+
.. code-block:: javascript
79+
:copyable: false
80+
81+
{
82+
saleTimestamp: Timestamp({ t: 1622731060, i: 1 }),
83+
saleIncrement: Long("1")
84+
},
85+
{
86+
saleTimestamp: Timestamp({ t: 1622731060, i: 2 }),
87+
saleIncrement: Long("2")
88+
},
89+
{
90+
saleTimestamp: Timestamp({ t: 1714124193, i: 1 }),
91+
saleIncrement: Long("1")
92+
},
93+
{
94+
saleTimestamp: Timestamp({ t: 1714124193, i: 2 }),
95+
saleIncrement: Long("2")
96+
},
97+
{
98+
saleTimestamp: Timestamp({ t: 1714124193, i: 3 }),
99+
saleIncrement: Long("3")
100+
}
101+
102+
Use ``$tsIncrement`` in a Change Stream Cursor to Monitor Collection Changes
103+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
104+
105+
The example in this section uses :expression:`$tsIncrement` in a
106+
:ref:`change stream cursor <changeStreams>` to return every other change
107+
made to a collection in the same second of time.
108+
109+
Create a :ref:`change stream cursor <changeStreams>` on a collection
110+
named ``cakeSales`` that you will see later in this section:
111+
112+
.. code-block:: javascript
113+
114+
cakeSalesCursor = db.cakeSales.watch( [
115+
{
116+
$match: {
117+
$expr: {
118+
$eq: [
119+
{ $mod: [ { $tsIncrement: "$clusterTime" } , 2 ] },
120+
0
121+
]
122+
}
123+
}
124+
}
125+
] )
126+
127+
In the example, the:
128+
129+
- :method:`db.collection.watch()` method creates a :ref:`change stream
130+
cursor <changeStreams>` for the ``cakeSales`` collection and stores
131+
the cursor in ``cakeSalesCursor``.
132+
133+
- :pipeline:`$match` stage filters the documents to those
134+
returned by the :query:`$expr` operator.
135+
136+
- :query:`$expr` operator:
137+
138+
- Applies :expression:`$mod` ``2`` to the ``$clusterTime`` variable's
139+
incrementing ordinal returned by :expression:`$tsIncrement`.
140+
141+
``$clusterTime`` is the timestamp from the :ref:`oplog
142+
<replica-set-oplog>` entry when the ``cakeSales`` collection is
143+
modified. See :ref:`Command Response <command-response>`.
144+
145+
- Compares the returned value from :expression:`$mod` to ``0`` using
146+
:expression:`$eq`.
147+
148+
.. include:: /includes/cakeSales-example-collection.rst
149+
150+
To monitor the ``cakeSales`` collection changes, use
151+
``cakeSalesCursor``. For example, to obtain the next document from
152+
``cakeSalesCursor``, use the :method:`~cursor.next()` method:
153+
154+
.. code-block:: javascript
155+
156+
cakeSalesCursor.next()
157+
158+
Depending on the second when the documents were added to ``cakeSales``,
159+
the output from ``cakeSalesCursor.next()`` varies. For example, the
160+
document additions might span more than one second.
161+
162+
The following ``cakeSalesCursor.next()`` example output shows the
163+
``insert`` details for the first document added to the ``cakeSales``
164+
collection. Notice the incrementing ordinal ``i`` is ``2`` in the
165+
``clusterTime`` field.
166+
167+
.. code-block:: javascript
168+
:copyable: false
169+
170+
_id: {
171+
_data: '82613A4F25000000022B022C0100296E5A100454C5BFAF538C47AB950614F43889BE00461E5F696400290004'
172+
},
173+
operationType: 'insert',
174+
clusterTime: Timestamp({ t: 1631211301, i: 2 }),
175+
fullDocument: {
176+
_id: 0,
177+
type: 'chocolate',
178+
orderDate: ISODate("2020-05-18T14:10:30.000Z"),
179+
state: 'CA',
180+
price: 13,
181+
quantity: 120
182+
},
183+
ns: { db: 'test', coll: 'cakeSales' },
184+
documentKey: { _id: 0 }
185+
186+
Running ``cakeSalesCursor.next()`` again returns the ``cakeSales``
187+
document for which the ``clusterTime`` incrementing ordinal ``i`` is
188+
``4``, omitting the document where ``i`` is ``3``.

0 commit comments

Comments
 (0)