Skip to content

Make PrimitiveJob serializable #12963

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Apr 2, 2025
Merged

Conversation

kt474
Copy link
Member

@kt474 kt474 commented Aug 15, 2024

Summary

In qiskit-ibm-runtime we want to add job methods to our QiskitRuntimeLocalService (Issue Qiskit/qiskit-ibm-runtime#1607). In order to save PrimitiveJob objects locally we need a way to serialize them. It's currently not serializable due to the use of ThreadPoolExecutor:

executor = ThreadPoolExecutor(max_workers=1) # pylint: disable=consider-using-with
self._future = executor.submit(self._function, *self._args, **self._kwargs)

Credit to @t-imamichi

Details and comments

Also fixes #12787

Unverified

The email in this signature doesn’t match the committer email.
@kt474 kt474 requested review from a team as code owners August 15, 2024 17:23
@qiskit-bot
Copy link
Collaborator

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core
  • @ajavadia
  • @levbishop
  • @t-imamichi

@coveralls
Copy link

coveralls commented Aug 15, 2024

Pull Request Test Coverage Report for Build 14136417982

Details

  • 22 of 25 (88.0%) changed or added relevant lines in 1 file are covered.
  • 5 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.03%) to 88.093%

Changes Missing Coverage Covered Lines Changed/Added Lines %
qiskit/primitives/primitive_job.py 22 25 88.0%
Files with Coverage Reduction New Missed Lines %
crates/accelerate/src/unitary_synthesis.rs 1 94.79%
crates/qasm2/src/expr.rs 1 94.23%
crates/qasm2/src/lex.rs 3 92.73%
Totals Coverage Status
Change from base Build 14129997569: 0.03%
Covered Lines: 72775
Relevant Lines: 82612

💛 - Coveralls

Unverified

The email in this signature doesn’t match the committer email.
@t-imamichi t-imamichi added mod: primitives Related to the Primitives module Changelog: New Feature Include in the "Added" section of the changelog labels Aug 16, 2024
kt474 added 2 commits August 16, 2024 12:55

Unverified

The email in this signature doesn’t match the committer email.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@kt474 kt474 requested a review from t-imamichi December 5, 2024 16:14
@t-imamichi
Copy link
Member

It looks good overall. Could you add a reno to describe that both DataBin and PrimitiveJob become serializable?

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Unverified

The email in this signature doesn’t match the committer email.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
t-imamichi
t-imamichi previously approved these changes Dec 11, 2024
Copy link
Member

@t-imamichi t-imamichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you ask someone else to review it too just in case?

Unverified

The email in this signature doesn’t match the committer email.

Unverified

The email in this signature doesn’t match the committer email.
…-terra into update-primitve-job

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Unverified

The email in this signature doesn’t match the committer email.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: Takashi Imamichi <[email protected]>
@t-imamichi t-imamichi requested a review from ihincks January 23, 2025 03:06
t-imamichi
t-imamichi previously approved these changes Mar 24, 2025
Copy link
Member

@t-imamichi t-imamichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you review it, @ihincks?

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
ihincks
ihincks previously approved these changes Mar 26, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Copy link
Contributor

@raynelfss raynelfss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removal of the unimplemented __setattr__ method makes sense to me, but I have a question about the _prepare_dump method.

Comment on lines 51 to 56
def _prepare_dump(self):
"""This method allows PrimitiveJob to be serialized"""
_ = self.result()
_ = self.status()
self._future = None

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When is this function called during serialization? I can't see anywhere where the method is called in the PR itself?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be called by the user if they want to serialize PrimitiveJob - example here

Copy link
Contributor

@raynelfss raynelfss Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to make this happen in one of the serialization methods that python has built-in? I believe this could be executed on one of the methods, avoiding that function call. You could include the lines here and override one of the reserved methods for serialization: __reduce__, __getstate__, __getnewargs__, depending on your needs.

Note: This might require you to also implement a __setstate__ method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your suggestion. I think the following should work (to remove non-pickable object _future) by replacing _prepare_dump with the following.

    def __getstate__(self):
        _ = self.result()
        _ = self.status()
        state = self.__dict__.copy()
        state["_future"] = None
        return state

    def __setstate__(self, state):
        self.__dict__.update(state)
        self._future = None

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a patch to this PR kt474#1.

t-imamichi and others added 2 commits March 27, 2025 14:33

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Implement __getstate__ and __setstate__
@kt474 kt474 dismissed stale reviews from ihincks and t-imamichi via 574d437 March 27, 2025 17:25

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@kt474 kt474 requested a review from raynelfss March 27, 2025 17:28
raynelfss
raynelfss previously approved these changes Mar 27, 2025
Copy link
Contributor

@raynelfss raynelfss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks much better! Thank you for these additions @kt474, @t-imamichi 🚀

Unverified

The email in this signature doesn’t match the committer email.
raynelfss
raynelfss previously approved these changes Mar 27, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: Takashi Imamichi <[email protected]>
kt474 and others added 2 commits March 28, 2025 15:24

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: Takashi Imamichi <[email protected]>

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@kt474 kt474 requested review from t-imamichi and raynelfss April 1, 2025 20:11
@raynelfss raynelfss added this pull request to the merge queue Apr 2, 2025
Merged via the queue into Qiskit:main with commit 5557271 Apr 2, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: New Feature Include in the "Added" section of the changelog mod: primitives Related to the Primitives module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make DataBin pickle-able
6 participants