Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firestore Spec Tests: Port JS PR 7229 (optimized query resumption using bloom filter) #4929

Merged
merged 5 commits into from
Apr 25, 2023

Conversation

dconeybe
Copy link
Contributor

@dconeybe dconeybe commented Apr 24, 2023

Port spec test changes from firebase/firebase-js-sdk#7229 (Optimize local cache sync when resuming a query that had docs deleted).

Note that the integer value of QueryPurpose.LIMBO_RESOLUTION changed from 2 to 3 in the spec test JSON files as a result of this port and, therefore, some custom logic was added into SpecTestCase.java to handle this correctly.

For reference, here is the iOS port of the same change: firebase/firebase-ios-sdk#11185

… readTime and resumeToken into a data structure)
…ocal cache sync when resuming a query that had docs deleted)
@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 24, 2023

Coverage Report 1

Affected Products

  • firebase-firestore

    Overall coverage changed from ? (1b43eb2) to 44.22% (aa20236) by ?.

    324 individual files with coverage change

    FilenameBase (1b43eb2)Merge (aa20236)Diff
    AbstractStream.java?34.81%?
    ActivityScope.java?0.00%?
    AggregateField.java?31.03%?
    AggregateQuery.java?22.73%?
    AggregateQuerySnapshot.java?59.52%?
    AggregateSource.java?0.00%?
    AggregationResult.java?0.00%?
    AggregationResultOrBuilder.java?0.00%?
    AggregationResultProto.java?0.00%?
    AndroidConnectivityMonitor.java?39.51%?
    ApiUtil.java?12.50%?
    ArrayContainsAnyFilter.java?100.00%?
    ArrayContainsFilter.java?100.00%?
    ArrayTransformOperation.java?86.49%?
    ArrayValue.java?48.60%?
    ArrayValueOrBuilder.java?0.00%?
    Assert.java?60.00%?
    AsyncEventListener.java?0.00%?
    AsyncQueue.java?70.65%?
    AutoValue_FieldIndex.java?58.33%?
    AutoValue_FieldIndex_IndexOffset.java?58.62%?
    AutoValue_FieldIndex_IndexState.java?59.09%?
    AutoValue_FieldIndex_Segment.java?54.17%?
    AutoValue_IndexEntry.java?34.21%?
    AutoValue_Overlay.java?54.55%?
    AutoValue_TestingHooks_ExistenceFilterMismatchInfo.java?20.00%?
    BackgroundQueue.java?81.25%?
    BasePath.java?86.54%?
    BatchGetDocumentsRequest.java?0.00%?
    BatchGetDocumentsRequestOrBuilder.java?0.00%?
    BatchGetDocumentsResponse.java?0.00%?
    BatchGetDocumentsResponseOrBuilder.java?0.00%?
    BeginTransactionRequest.java?0.00%?
    BeginTransactionRequestOrBuilder.java?0.00%?
    BeginTransactionResponse.java?0.00%?
    BeginTransactionResponseOrBuilder.java?0.00%?
    Blob.java?76.92%?
    Bound.java?39.29%?
    BundleCache.java?0.00%?
    BundleCallback.java?0.00%?
    BundledDocumentMetadata.java?0.00%?
    BundledDocumentMetadataOrBuilder.java?0.00%?
    BundleDocument.java?90.00%?
    BundledQuery.java?34.57%?
    BundledQueryOrBuilder.java?0.00%?
    BundleElement.java?0.00%?
    BundleElementOrBuilder.java?0.00%?
    BundleLoader.java?100.00%?
    BundleMetadata.java?0.00%?
    BundleMetadataOrBuilder.java?0.00%?
    BundleProto.java?0.00%?
    BundleReader.java?95.29%?
    BundleSerializer.java?89.52%?
    ByteBufferInputStream.java?83.33%?
    CollectionReference.java?13.64%?
    CommitRequest.java?0.00%?
    CommitRequestOrBuilder.java?0.00%?
    CommitResponse.java?0.00%?
    CommitResponseOrBuilder.java?0.00%?
    CommonProto.java?0.00%?
    ComponentProvider.java?100.00%?
    CompositeFilter.java?98.57%?
    ConnectivityMonitor.java?0.00%?
    Consumer.java?0.00%?
    CreateDocumentRequest.java?0.00%?
    CreateDocumentRequestOrBuilder.java?0.00%?
    CredentialsProvider.java?100.00%?
    Cursor.java?35.29%?
    CursorOrBuilder.java?0.00%?
    CustomClassMapper.java?84.25%?
    DatabaseId.java?70.37%?
    DatabaseInfo.java?90.91%?
    Datastore.java?26.47%?
    DatastoreTestTrace.java?0.00%?
    DeleteDocumentRequest.java?0.00%?
    DeleteDocumentRequestOrBuilder.java?0.00%?
    DeleteMutation.java?90.48%?
    DirectionalIndexByteEncoder.java?100.00%?
    Document.java?100.00%?
    DocumentChange.java?67.61%?
    DocumentChangeOrBuilder.java?0.00%?
    DocumentCollections.java?83.33%?
    DocumentDelete.java?34.62%?
    DocumentDeleteOrBuilder.java?0.00%?
    DocumentId.java?0.00%?
    DocumentKey.java?97.14%?
    DocumentMask.java?40.66%?
    DocumentMaskOrBuilder.java?0.00%?
    DocumentOrBuilder.java?0.00%?
    DocumentOverlayCache.java?0.00%?
    DocumentProto.java?0.00%?
    DocumentReference.java?13.24%?
    DocumentRemove.java?29.23%?
    DocumentRemoveOrBuilder.java?0.00%?
    DocumentSet.java?83.78%?
    DocumentSnapshot.java?37.50%?
    DocumentTransform.java?29.31%?
    DocumentTransformOrBuilder.java?0.00%?
    DocumentViewChange.java?90.91%?
    DocumentViewChangeSet.java?87.88%?
    EncodedPath.java?93.22%?
    EventListener.java?0.00%?
    EventManager.java?96.15%?
    Exclude.java?0.00%?
    Executors.java?100.00%?
    ExistenceFilter.java?80.00%?
    ExistenceFilterOrBuilder.java?0.00%?
    ExponentialBackoff.java?30.43%?
    FieldFilter.java?93.06%?
    FieldIndex.java?98.18%?
    FieldMask.java?57.89%?
    FieldPath.java?89.66%?
    FieldTransform.java?64.71%?
    FieldValue.java?82.14%?
    FileUtil.java?0.00%?
    Filter.java?0.00%?
    FirebaseAppCheckTokenProvider.java?93.75%?
    FirebaseAuthCredentialsProvider.java?100.00%?
    FirebaseClientGrpcMetadataProvider.java?44.00%?
    FirebaseFirestore.java?38.33%?
    FirebaseFirestoreException.java?83.72%?
    FirebaseFirestoreSettings.java?71.91%?
    FirestoreCallCredentials.java?18.60%?
    FirestoreChannel.java?18.10%?
    FirestoreClient.java?36.11%?
    FirestoreGrpc.java?2.46%?
    FirestoreIndexValueWriter.java?77.11%?
    FirestoreMultiDbComponent.java?100.00%?
    FirestoreProto.java?0.00%?
    FirestoreRegistrar.java?100.00%?
    Function.java?0.00%?
    GeoPoint.java?91.67%?
    GetDocumentRequest.java?0.00%?
    GetDocumentRequestOrBuilder.java?0.00%?
    GrpcCallProvider.java?69.41%?
    GrpcMetadataProvider.java?0.00%?
    IgnoreExtraProperties.java?0.00%?
    IncomingStreamObserver.java?0.00%?
    Index.java?30.02%?
    IndexBackfiller.java?100.00%?
    IndexByteEncoder.java?88.57%?
    IndexEntry.java?100.00%?
    IndexManager.java?100.00%?
    IndexOrBuilder.java?0.00%?
    IndexProto.java?0.00%?
    InFilter.java?100.00%?
    IntMath.java?41.67%?
    KeyFieldFilter.java?66.67%?
    KeyFieldInFilter.java?0.00%?
    KeyFieldNotInFilter.java?0.00%?
    LimboDocumentChange.java?70.59%?
    ListCollectionIdsRequest.java?0.00%?
    ListCollectionIdsRequestOrBuilder.java?0.00%?
    ListCollectionIdsResponse.java?0.00%?
    ListCollectionIdsResponseOrBuilder.java?0.00%?
    ListDocumentsRequest.java?0.00%?
    ListDocumentsRequestOrBuilder.java?0.00%?
    ListDocumentsResponse.java?0.00%?
    ListDocumentsResponseOrBuilder.java?0.00%?
    Listener.java?0.00%?
    ListenerRegistration.java?0.00%?
    ListenerRegistrationImpl.java?0.00%?
    ListenRequest.java?10.66%?
    ListenRequestOrBuilder.java?0.00%?
    ListenResponse.java?32.64%?
    ListenResponseOrBuilder.java?0.00%?
    ListenSequence.java?100.00%?
    LoadBundleTask.java?25.26%?
    LoadBundleTaskProgress.java?60.47%?
    LocalCacheSettings.java?0.00%?
    LocalDocumentsResult.java?100.00%?
    LocalDocumentsView.java?100.00%?
    LocalSerializer.java?97.35%?
    LocalStore.java?100.00%?
    LocalViewChanges.java?100.00%?
    Logger.java?91.67%?
    LogicUtils.java?96.67%?
    LruDelegate.java?0.00%?
    LruGarbageCollector.java?93.64%?
    MapValue.java?53.00%?
    MapValueOrBuilder.java?0.00%?
    MemoryBundleCache.java?100.00%?
    MemoryCacheSettings.java?50.00%?
    MemoryComponentProvider.java?82.93%?
    MemoryDocumentOverlayCache.java?100.00%?
    MemoryEagerGcSettings.java?28.57%?
    MemoryEagerReferenceDelegate.java?100.00%?
    MemoryGarbageCollectorSettings.java?0.00%?
    MemoryIndexManager.java?68.97%?
    MemoryLruGcSettings.java?0.00%?
    MemoryLruReferenceDelegate.java?96.00%?
    MemoryMutationQueue.java?100.00%?
    MemoryOverlayMigrationManager.java?100.00%?
    MemoryPersistence.java?100.00%?
    MemoryRemoteDocumentCache.java?98.28%?
    MemoryTargetCache.java?100.00%?
    MetadataChanges.java?100.00%?
    MutableDocument.java?100.00%?
    Mutation.java?100.00%?
    MutationBatch.java?89.23%?
    MutationBatchResult.java?100.00%?
    MutationQueue.java?0.00%?
    MutationResult.java?100.00%?
    NamedQuery.java?0.00%?
    NamedQueryOrBuilder.java?0.00%?
    NotInFilter.java?85.71%?
    NumberComparisonHelper.java?100.00%?
    NumericIncrementTransformOperation.java?87.50%?
    ObjectValue.java?99.06%?
    OnlineState.java?100.00%?
    OnlineStateTracker.java?98.11%?
    OnProgressListener.java?0.00%?
    OrderBy.java?96.55%?
    OrderedCodeWriter.java?68.52%?
    Overlay.java?100.00%?
    OverlayedDocument.java?100.00%?
    OverlayMigrationManager.java?0.00%?
    PatchMutation.java?100.00%?
    Persistence.java?100.00%?
    PersistentCacheSettings.java?62.50%?
    Precondition.java?37.82%?
    PreconditionOrBuilder.java?0.00%?
    Preconditions.java?57.89%?
    PropertyName.java?0.00%?
    Query.java?3.96%?
    QueryDocumentSnapshot.java?64.71%?
    QueryEngine.java?98.63%?
    QueryListener.java?100.00%?
    QueryProto.java?0.00%?
    QueryPurpose.java?100.00%?
    QueryResult.java?100.00%?
    QuerySnapshot.java?76.36%?
    QueryView.java?100.00%?
    ReferenceDelegate.java?0.00%?
    ReferenceSet.java?92.45%?
    RemoteDocumentCache.java?0.00%?
    RemoteEvent.java?92.31%?
    RemoteSerializer.java?80.57%?
    RemoteStore.java?88.49%?
    ResourcePath.java?94.74%?
    RollbackRequest.java?0.00%?
    RollbackRequestOrBuilder.java?0.00%?
    RunAggregationQueryRequest.java?0.00%?
    RunAggregationQueryRequestOrBuilder.java?0.00%?
    RunAggregationQueryResponse.java?0.00%?
    RunAggregationQueryResponseOrBuilder.java?0.00%?
    RunQueryRequest.java?0.00%?
    RunQueryRequestOrBuilder.java?0.00%?
    RunQueryResponse.java?0.00%?
    RunQueryResponseOrBuilder.java?0.00%?
    Scheduler.java?0.00%?
    ServerTimestamp.java?0.00%?
    ServerTimestampOperation.java?100.00%?
    ServerTimestamps.java?96.00%?
    SetMutation.java?94.44%?
    SetOptions.java?0.00%?
    SnapshotMetadata.java?68.75%?
    SnapshotVersion.java?87.50%?
    Source.java?0.00%?
    SQLiteBundleCache.java?90.00%?
    SQLiteComponentProvider.java?100.00%?
    SQLiteDocumentOverlayCache.java?97.85%?
    SQLiteIndexManager.java?99.49%?
    SQLiteLruReferenceDelegate.java?98.67%?
    SQLiteMutationQueue.java?82.76%?
    SQLiteOverlayMigrationManager.java?90.48%?
    SQLitePersistence.java?86.34%?
    SQLiteRemoteDocumentCache.java?98.21%?
    SQLiteSchema.java?96.63%?
    SQLiteTargetCache.java?98.47%?
    Stream.java?100.00%?
    StructuredAggregationQuery.java?0.00%?
    StructuredAggregationQueryOrBuilder.java?0.00%?
    StructuredQuery.java?32.54%?
    StructuredQueryOrBuilder.java?0.00%?
    Supplier.java?0.00%?
    SyncEngine.java?93.31%?
    Target.java?95.79%?
    TargetCache.java?0.00%?
    TargetChange.java?80.00%?
    TargetChangeOrBuilder.java?0.00%?
    TargetData.java?77.50%?
    TargetIdGenerator.java?100.00%?
    TargetIndexMatcher.java?100.00%?
    TargetOrBuilder.java?0.00%?
    TargetState.java?97.78%?
    TestingHooks.java?40.91%?
    ThrottledForwardingExecutor.java?100.00%?
    ThrowOnExtraProperties.java?0.00%?
    Timestamp.java?80.39%?
    Token.java?0.00%?
    Transaction.java?0.00%?
    TransactionOptions.java?0.00%?
    TransactionOptionsOrBuilder.java?0.00%?
    TransactionRunner.java?0.00%?
    TransformOperation.java?0.00%?
    UpdateDocumentRequest.java?0.00%?
    UpdateDocumentRequestOrBuilder.java?0.00%?
    User.java?78.57%?
    UserData.java?66.00%?
    UserDataReader.java?70.48%?
    UserDataWriter.java?51.02%?
    Util.java?67.63%?
    Value.java?41.22%?
    ValueOrBuilder.java?0.00%?
    Values.java?95.65%?
    VerifyMutation.java?46.15%?
    View.java?96.82%?
    ViewChange.java?100.00%?
    ViewSnapshot.java?89.23%?
    WatchChange.java?70.51%?
    WatchChangeAggregator.java?98.26%?
    WatchStream.java?19.23%?
    Write.java?36.11%?
    WriteBatch.java?0.00%?
    WriteOrBuilder.java?0.00%?
    WriteProto.java?0.00%?
    WriteRequest.java?10.41%?
    WriteRequestOrBuilder.java?0.00%?
    WriteResponse.java?11.70%?
    WriteResponseOrBuilder.java?0.00%?
    WriteResult.java?0.00%?
    WriteResultOrBuilder.java?0.00%?
    WriteStream.java?31.82%?

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/3qozOvlYnE.html

@github-actions
Copy link
Contributor

github-actions bot commented Apr 24, 2023

Unit Test Results

   160 files  ±0     160 suites  ±0   2m 49s ⏱️ +41s
1 125 tests ±0  1 109 ✔️ ±0  16 💤 ±0  0 ±0 
2 250 runs  ±0  2 218 ✔️ ±0  32 💤 ±0  0 ±0 

Results for commit fa399de. ± Comparison against base commit 1b43eb2.

♻️ This comment has been updated with latest results.

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 24, 2023

Size Report 1

Affected Products

  • base

    TypeBase (1b43eb2)Merge (aa20236)Diff
    apk (aggressive)?8.39 kB? (?)
    apk (release)?8.65 kB? (?)
  • firebase-appcheck-interop

    TypeBase (1b43eb2)Merge (aa20236)Diff
    aar?5.00 kB? (?)
    apk (aggressive)?312 kB? (?)
    apk (release)?931 kB? (?)
  • firebase-firestore

    TypeBase (1b43eb2)Merge (aa20236)Diff
    aar?1.33 MB? (?)
    apk (aggressive)?518 kB? (?)
    apk (release)?3.94 MB? (?)
  • protolite-well-known-types

    TypeBase (1b43eb2)Merge (aa20236)Diff
    aar?999 kB? (?)
    apk (aggressive)?134 kB? (?)
    apk (release)?666 kB? (?)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/dgz1TfbGGI.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 24, 2023

Startup Time Report 1

Note: Layout is sometimes suboptimal due to limited formatting support on GitHub. Please check this report on GCS.

Startup time comparison between the CI merge commit (aa20236) and the base commit (1b43eb2) are not available.

No macrobenchmark data found for the base commit (1b43eb2). Analysis for the CI merge commit (aa20236) can be found at:

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/j91pMYpRIe/index.html

@dconeybe dconeybe changed the title Port spec test changes from https://github.com/firebase/firebase-js-s… Firestore Spec Tests: Port JS PR 7229 (optimized query resumption using bloom filter) Apr 24, 2023
…dkPr7021' into SpecTestPortJsSdkPr7229_PR4929
Base automatically changed from dconeybe/SpecTestPortJsSdkPr7021 to master April 25, 2023 14:18
@tom-andersen tom-andersen self-requested a review April 25, 2023 15:48
Copy link
Contributor

@tom-andersen tom-andersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants