Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added cardano connector plugin #1636

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

SupernaviX
Copy link
Contributor

@SupernaviX SupernaviX commented Feb 8, 2025

Proposed changes

Adds a cardano connector plugin to firefly core. This connector interacts with the firefly-cardanoconnect service from firefly-cardano.


Types of changes

  • Bug fix
  • New feature added
  • Documentation Update

Please make sure to follow these points

  • I have read the contributing guidelines.
  • I have performed a self-review of my own code or work.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generates no new warnings. (uh
  • My Pull Request title is in format < issue name > eg Added links in the documentation.

firefly


Other Information

The Cardano connector and implementation is still a work in progress; in particular we need documentation and tests. However, it works well enough to demo, and I'd like to merge this relatively soon so that people can run it without building firefly-core from a branch, or downloading an image from a private repo.

@SupernaviX SupernaviX requested a review from a team as a code owner February 8, 2025 00:13
Signed-off-by: Simon Gellis <[email protected]>
Signed-off-by: Simon Gellis <[email protected]>
Copy link

codecov bot commented Feb 8, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.96%. Comparing base (7b7dbb0) to head (7321ba9).

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #1636    +/-   ##
========================================
  Coverage   99.95%   99.96%            
========================================
  Files         339      342     +3     
  Lines       29782    30457   +675     
========================================
+ Hits        29770    30445   +675     
  Misses          8        8            
  Partials        4        4            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@EnriqueL8 EnriqueL8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! It's fantastic to see 👏🏼

Some initial thoughts and will follow up with more thorough review after playing around with it 😃

Address string `json:"address"`
}

var addressVerify = regexp.MustCompile("^addr1|^addr_test1|^stake1|^stake_test1")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just used for (extremely simple) address validation. Cardano uses bech32 addresses with a conventional set of HRPs, but I wanted to avoid pulling in a bech32 decoder, so I allowlisted the conventional HRPs instead.

Comment on lines 390 to 393
func (c *Cardano) GenerateErrorSignature(ctx context.Context, event *fftypes.FFIErrorDefinition) string {
// TODO: impl
return ""
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be interested to understand more how errors are reported in Cardano from smart contract

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cardano smart contract doesn't report errors right now, only events. I don't understand the difference between errors and events, and I couldn't find any code in other plugins which streamed errors from the connector.

I've copied the "event signature" code into this function, but that's just a guess at what it's supposed to do. I'd want to see how consumers actually use events to be more confident about that.

Comment on lines 542 to 550
if !isBatch {
var receipt common.BlockchainReceiptNotification
_ = json.Unmarshal(msgBytes, &receipt)

err := common.HandleReceipt(ctx, "", c, &receipt, c.callbacks)
if err != nil {
l.Errorf("Failed to process receipt: %+v", msgTyped)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that this is heavily inspired from FFTM and the evm plugin - I would recommend in the future moving to a durable event stream here where the receipts have to be ack'd back as part of a batch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using EVM, Tezos, and FFTM as references for how the eventstream is supposed to work (trying my best to make Cardano match the semantics of other connectors). It looked like other connectors assume that every message in a batch is an event, not a receipt.

I think it makes sense to use resilient ack-ing for everything, but I'm a little wary about inventing semantics for it in this plugin that nothing else is using.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - probably as a follow up. I'm building a framework at moment to allow for resilient ack-ing and get confirmation from FireFly core it has processed things correctly

Comment on lines 564 to 582
switch msgJSON.GetString("type") {
case "ContractEvent":
signature := msgJSON.GetString("signature")

logger := log.L(ctx)
logger.Infof("[%d:%d/%d]: '%s'", batchID, i+1, count, signature)
logger.Tracef("Message: %+v", msgJSON)
c.processContractEvent(ctx, namespace, events, msgJSON)
case "Receipt":
var receipt common.BlockchainReceiptNotification
msgBytes, _ := json.Marshal(msgMap)
_ = json.Unmarshal(msgBytes, &receipt)

err := common.HandleReceipt(ctx, namespace, c, &receipt, c.callbacks)
if err != nil {
log.L(ctx).Errorf("Failed to process receipt: %+v", msgMap)
}
default:
log.L(ctx).Errorf("Unexpected message in batch: %+v", msgMap)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome, thanks!

@EnriqueL8
Copy link
Contributor

EnriqueL8 commented Mar 12, 2025

Error from build - FYI @onelapahead I believe you added recently this DX callback. Feels like a race condition as sometimes fails and others passes

2025-03-11T23:25:22.9354359Z === RUN   TestEventsWithManifest
2025-03-11T23:25:22.9354697Z time="2025-03-11T23:25:13Z" level=info msg="WS ws://127.0.0.1:37865/ws connected"
2025-03-11T23:25:22.9355572Z time="2025-03-11T23:25:13Z" level=error msg="Message cannot be parsed as JSON: invalid character '!' looking for beginning of value\n!}" dx=https role=event-loop
2025-03-11T23:25:22.9356408Z time="2025-03-11T23:25:13Z" level=warning msg="Failed to dispatch DX event: FF10380: Unexpected websocket event type from DX plugin: " dx=https
2025-03-11T23:25:22.9356819Z     ffdx_test.go:817: PASS:	OperationUpdate(mock.argumentMatcher)
2025-03-11T23:25:22.9357594Z     ffdx_test.go:817: PASS:	OperationUpdate(mock.argumentMatcher)
2025-03-11T23:25:22.9358206Z time="2025-03-11T23:25:13Z" level=info msg="WS ws://127.0.0.1:37865/ws closed: websocket: close 1006 (abnormal closure): unexpected EOF"
2025-03-11T23:25:22.9358329Z panic: 
2025-03-11T23:25:22.9358671Z assert: mock: I don't know what to return because the method call was unexpected.
2025-03-11T23:25:22.9359055Z 	Either do Mock.On("DXConnect").Return(...) first, or remove the DXConnect() call.
2025-03-11T23:25:22.9359216Z 	This method was unexpected:
2025-03-11T23:25:22.9359367Z 		DXConnect(*ffdx.FFDX)
2025-03-11T23:25:22.9364836Z 		0: &ffdx.FFDX{ctx:(*context.valueCtx)(0xc000446ba0), cancelCtx:(context.CancelFunc)(0x552540), capabilities:(*dataexchange.Capabilities)(0xc0004beb1f), callbacks:ffdx.callbacks{plugin:(*ffdx.FFDX)(0xc00012adc0), writeLock:sync.Mutex{state:0, sema:0x0}, handlers:map[string]dataexchange.Callbacks{"ns1:node1":(*dataexchangemocks.Callbacks)(0xc000288cd0)}, opHandlers:map[string]core.OperationCallbacks{"ns1":(*coremocks.OperationCallbacks)(0xc000288d20)}}, client:(*resty.Client)(0xc0004fac88), wsconn:(*wsclient.wsClient)(0xc0005610a0), needsInit:false, initialized:true, initMutex:sync.Mutex{state:1, sema:0x0}, nodes:map[string]*ffdx.dxNode{}, ackChannel:(chan *ffdx.ack)(0xc0004ce600), retry:(*retry.Retry)(0xc000308180), backgroundStart:false, backgroundRetry:(*retry.Retry)(nil), metrics:(*metricsmocks.Manager)(0xc00008a6e0)}
2025-03-11T23:25:22.9369728Z 	at: [/home/runner/work/firefly/firefly/mocks/dataexchangemocks/callbacks.go:17 /home/runner/work/firefly/firefly/internal/dataexchange/ffdx/ffdx.go:311 /home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:301 /home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/retry/retry.go:57 /home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/retry/retry.go:42 /home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:294 /home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:497 /opt/hostedtoolcache/go/1.22.12/x64/src/runtime/asm_amd64.s:1695]
2025-03-11T23:25:22.9369747Z 
2025-03-11T23:25:22.9369898Z goroutine 251 [running]:
2025-03-11T23:25:22.9370413Z github.com/stretchr/testify/mock.(*Mock).fail(0xc000288cd0, {0xa50de6?, 0x8?}, {0xc0003cb500?, 0x1?, 0x1?})
2025-03-11T23:25:22.9370855Z 	/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/mock/mock.go:332 +0x134
2025-03-11T23:25:22.9371388Z github.com/stretchr/testify/mock.(*Mock).MethodCalled(0xc000288cd0, {0xbbdc9e, 0x9}, {0xc0002eaf80, 0x1, 0x1})
2025-03-11T23:25:22.9371843Z 	/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/mock/mock.go:499 +0x705
2025-03-11T23:25:22.9372335Z github.com/stretchr/testify/mock.(*Mock).Called(0xc000288cd0, {0xc0002eaf80, 0x1, 0x1})
2025-03-11T23:25:22.9372776Z 	/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/mock/mock.go:464 +0x137
2025-03-11T23:25:22.9373431Z github.com/hyperledger/firefly/mocks/dataexchangemocks.(*Callbacks).DXConnect(0xc000288cd0, {0xb18e98, 0xc00012adc0})
2025-03-11T23:25:22.9373913Z 	/home/runner/work/firefly/firefly/mocks/dataexchangemocks/callbacks.go:17 +0x7a
2025-03-11T23:25:22.9374704Z github.com/hyperledger/firefly/internal/dataexchange/ffdx.(*FFDX).beforeConnect(0xc00012adc0, {0xb134d0, 0xc00008a730}, {0x751820?, 0xc0001de9a0?})
2025-03-11T23:25:22.9375146Z 	/home/runner/work/firefly/firefly/internal/dataexchange/ffdx/ffdx.go:311 +0x779
2025-03-11T23:25:22.9375554Z github.com/hyperledger/firefly-common/pkg/wsclient.(*wsClient).connect.func1(0x1)
2025-03-11T23:25:22.9376147Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:301 +0x82
2025-03-11T23:25:22.9376850Z github.com/hyperledger/firefly-common/pkg/retry.(*Retry).Do(0xc0005610e0, {0xb134d0, 0xc00008a730}, {0x0, 0x0}, 0xc00042fee0)
2025-03-11T23:25:22.9401500Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/retry/retry.go:57 +0xd6
2025-03-11T23:25:22.9401929Z github.com/hyperledger/firefly-common/pkg/retry.(*Retry).DoCustomLog(...)
2025-03-11T23:25:22.9402488Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/retry/retry.go:42
2025-03-11T23:25:22.9402962Z github.com/hyperledger/firefly-common/pkg/wsclient.(*wsClient).connect(0xc0005610a0, 0x0)
2025-03-11T23:25:22.9433341Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:294 +0x8b
2025-03-11T23:25:22.9433964Z github.com/hyperledger/firefly-common/pkg/wsclient.(*wsClient).receiveReconnectLoop(0xc0005610a0)
2025-03-11T23:25:22.9434649Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:497 +0x325
2025-03-11T23:25:22.9435224Z created by github.com/hyperledger/firefly-common/pkg/wsclient.(*wsClient).Connect in goroutine 245
2025-03-11T23:25:22.9435875Z 	/home/runner/go/pkg/mod/github.com/hyperledger/[email protected]/pkg/wsclient/wsclient.go:191 +0x65
2025-03-11T23:25:22.9436519Z FAIL	github.com/hyperledger/firefly/internal/dataexchange/ffdx	0.054s
2025-03-11T23:25:22.9436717Z === RUN   TestHandleFFIBroadcastOk

@EnriqueL8
Copy link
Contributor

Alright fixed above #1658

Copy link
Contributor

@EnriqueL8 EnriqueL8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was reminded yesterday of the key being mandatory today as part of deploying a contract and it doesn't make sense for Cardano as the connector has used that API to run a off-chain dApp that is responsible of building Cardano TX and indexing state. Raised a discussion on Discord https://discord.com/channels/905194001349627914/928377875827154984/1349704667524763689


## Create the stack

A Cardano stack can be run in two different ways; with a firefly
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A Cardano stack can be run in two different ways; with a firefly
A Cardano stack can be run in two different ways with Hyperledger FireFly


> **NOTE**: The cardano-node communicates over a Unix socket, so this will not work on Windows.

Start a local cardano node. The fastest way to do this is to [use mithril](https://mithril.network/doc/manual/getting-started/bootstrap-cardano-node/) to bootstrap the node.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Start a local cardano node. The fastest way to do this is to [use mithril](https://mithril.network/doc/manual/getting-started/bootstrap-cardano-node/) to bootstrap the node.
Start a local Cardano node. The fastest way to do this is to [use mithril](https://mithril.network/doc/manual/getting-started/bootstrap-cardano-node/) to bootstrap the node.

Comment on lines +73 to +90
## Get some ADA

Now that you have a stack, you need some seed funds to get started. Your stack was created with a wallet already (these are free to create in Cardano). To get the address, you can run
```sh
ff accounts list dev
```

The response will look like
```json
[
{
"address": "addr_test1...",
"privateKey": "..."
}
]
```

If you're developing against a testnet such as preview, you can receive funds from the [testnet faucet](https://docs.cardano.org/cardano-testnets/tools/faucet). Pass the `address` from that response to the faucet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would recommend following this a Transaction either in this section or working with smart contracts section. I do see a gap of how do you create a simple transfer of ADA through the API

@EnriqueL8
Copy link
Contributor

EnriqueL8 commented Mar 13, 2025

Sorry I tagged a comment in a linked PR as Fixes and it automatically closed this PR when the fix was merged

@EnriqueL8
Copy link
Contributor

Timeout error in a Cardano unit test

2025-03-15T04:08:58.2090672Z time="2025-03-15T04:08:13Z" level=info msg="Event stream: es12345 (topic=topic1/ns1)" proto=cardano
2025-03-15T04:08:58.2091695Z time="2025-03-15T04:08:13Z" level=info msg="WS ws://127.0.0.1:38187/ws connected" proto=cardano
2025-03-15T04:08:58.2092715Z time="2025-03-15T04:08:13Z" level=error msg="Message unexpected: map[bad:receipt]" proto=cardano role=event-loop
2025-03-15T04:08:58.2093959Z time="2025-03-15T04:08:13Z" level=error msg="Message cannot be parsed as JSON: different kind of bad batch" proto=cardano role=event-loop
2025-03-15T04:08:58.2095178Z time="2025-03-15T04:08:13Z" level=error msg="Unexpected message in batch: map[]" proto=cardano role=event-loop
2025-03-15T04:08:58.2096586Z time="2025-03-15T04:08:13Z" level=error msg="Message cannot be parsed as JSON: invalid character '!' looking for beginning of value\n!json" proto=cardano role=event-loop
2025-03-15T04:08:58.2098375Z time="2025-03-15T04:08:13Z" level=error msg="Message unexpected: map[not:a reply]" proto=cardano role=event-loop
2025-03-15T04:08:58.2099475Z time="2025-03-15T04:08:13Z" level=error msg="Message unexpected: 42" proto=cardano role=event-loop
2025-03-15T04:08:58.2100729Z time="2025-03-15T04:08:13Z" level=info msg="WS ws://127.0.0.1:38187/ws closed: websocket: close 1006 (abnormal closure): unexpected EOF" proto=cardano
2025-03-15T04:08:58.2102030Z time="2025-03-15T04:08:13Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 1 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2103411Z time="2025-03-15T04:08:13Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 2 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2104487Z time="2025-03-15T04:08:13Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 3 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2105552Z time="2025-03-15T04:08:14Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 4 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2106606Z time="2025-03-15T04:08:16Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 5 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2107893Z time="2025-03-15T04:08:20Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 6 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2108945Z time="2025-03-15T04:08:28Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 7 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2109990Z time="2025-03-15T04:08:44Z" level=warning msg="WS ws://127.0.0.1:38187/ws connect attempt 8 failed [-1]: " proto=cardano
2025-03-15T04:08:58.2113860Z coverage: 29.3% of statements
2025-03-15T04:08:58.2114274Z panic: test timed out after 45s
2025-03-15T04:08:58.2114707Z running tests:
2025-03-15T04:08:58.2115101Z 	TestInitAndStartWithCardanoConnect (45s)
2025-03-15T04:08:58.2115440Z 
2025-03-15T04:08:58.2115582Z goroutine 26 [running]:
2025-03-15T04:08:58.2115949Z testing.(*M).startAlarm.func1()
2025-03-15T04:08:58.2116605Z 	/opt/hostedtoolcache/go/1.22.12/x64/src/testing/testing.go:2366 +0x385
2025-03-15T04:08:58.2117428Z created by time.goFunc
2025-03-15T04:08:58.2117937Z 	/opt/hostedtoolcache/go/1.22.12/x64/src/time/sleep.go:177 +0x2d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants