stacker.news/api/paidAction/README.md

351 lines
18 KiB
Markdown
Raw Normal View History

backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
# Paid Actions
Paid actions are actions that require payments to perform. Given that we support several payment flows, some of which require more than one round of communication either with LND or the client, and several paid actions, we have this plugin-like interface to easily add new paid actions.
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
<details>
<summary>internals</summary>
All paid action progress, regardless of flow, is managed using a state machine that's transitioned by the invoice progress and payment progress (in the case of p2p paid action). Below is the full state machine for paid actions:
```mermaid
stateDiagram-v2
[*] --> PENDING
PENDING --> PAID
PENDING --> CANCELING
PENDING --> FAILED
PAID --> [*]
CANCELING --> FAILED
FAILED --> RETRYING
FAILED --> [*]
RETRYING --> [*]
[*] --> PENDING_HELD
PENDING_HELD --> HELD
PENDING_HELD --> FORWARDING
PENDING_HELD --> CANCELING
PENDING_HELD --> FAILED
HELD --> PAID
HELD --> CANCELING
HELD --> FAILED
FORWARDING --> FORWARDED
FORWARDING --> FAILED_FORWARD
FORWARDED --> PAID
FAILED_FORWARD --> CANCELING
FAILED_FORWARD --> FAILED
```
</details>
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
## Payment Flows
There are three payment flows:
### Fee credits
The stacker has enough fee credits to pay for the action. This is the simplest flow and is similar to a normal request.
### Optimistic
The optimistic flow is useful for actions that require immediate feedback to the client, but don't require the action to be immediately visible to everyone else.
For paid actions that support it, if the stacker doesn't have enough fee credits, we store the action in a `PENDING` state on the server, which is visible only to the stacker, then return a payment request to the client. The client then pays the invoice however and whenever they wish, and the server monitors payment progress. If the payment succeeds, the action is executed fully becoming visible to everyone and is marked as `PAID`. Otherwise, the action is marked as `FAILED`, the client is notified the payment failed and the payment can be retried.
<details>
<summary>Internals</summary>
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
Internally, optimistic flows make use of a state machine that's transitioned by the invoice payment progress.
```mermaid
stateDiagram-v2
[*] --> PENDING
PENDING --> PAID
PENDING --> CANCELING
PENDING --> FAILED
PAID --> [*]
CANCELING --> FAILED
FAILED --> RETRYING
FAILED --> [*]
RETRYING --> [*]
```
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
</details>
### Pessimistic
For paid actions that don't support optimistic actions (or when the stacker is `@anon`), if the client doesn't have enough fee credits, we return a payment request to the client without performing the action and only storing the action's arguments. After the client pays the invoice, the server performs the action with original arguments. Pessimistic actions require the payment to complete before being visible to them and everyone else.
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
Internally, pessimistic flows use hold invoices. If the action doesn't succeed, the payment is cancelled and it's as if the payment never happened (ie it's a lightning native refund mechanism).
<details>
<summary>Internals</summary>
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
Internally, pessimistic flows make use of a state machine that's transitioned by the invoice payment progress much like optimistic flows, but with extra steps.
```mermaid
stateDiagram-v2
PAID --> [*]
CANCELING --> FAILED
FAILED --> [*]
[*] --> PENDING_HELD
PENDING_HELD --> HELD
PENDING_HELD --> CANCELING
PENDING_HELD --> FAILED
HELD --> PAID
HELD --> CANCELING
HELD --> FAILED
```
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
</details>
### Table of existing paid actions and their supported flows
| action | fee credits | optimistic | pessimistic | anonable | qr payable | p2p wrapped | side effects |
| ----------------- | ----------- | ---------- | ----------- | -------- | ---------- | ----------- | ------------ |
| zaps | x | x | x | x | x | x | x |
| posts | x | x | x | x | x | | x |
| comments | x | x | x | x | x | | x |
| downzaps | x | x | | | x | | x |
| poll votes | x | x | | | x | | |
| territory actions | x | | x | | x | | |
| donations | x | | x | x | x | | |
| update posts | x | | x | | x | | x |
| update comments | x | | x | | x | | x |
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
## Not-custodial zaps (ie p2p wrapped payments)
Zaps, and possibly other future actions, can be performed peer to peer and non-custodially. This means that the payment is made directly from the client to the recipient, without the server taking custody of the funds. Currently, in order to trigger this behavior, the recipient must have a receiving wallet attached and the sender must have insufficient funds in their custodial wallet to perform the requested zap.
This works by requesting an invoice from the recipient's wallet and reusing the payment hash in a hold invoice paid to SN (to collect the sybil fee) which we serve to the sender. When the sender pays this wrapped invoice, we forward our own money to the recipient, who then reveals the preimage to us, allowing us to settle the wrapped invoice and claim the sender's funds. This effectively does what a lightning node does when forwarding a payment but allows us to do it at the application layer.
<details>
<summary>Internals</summary>
Internally, p2p wrapped payments make use of the same paid action state machine but it's transitioned by both the incoming invoice payment progress *and* the outgoing invoice payment progress.
```mermaid
stateDiagram-v2
PAID --> [*]
CANCELING --> FAILED
FAILED --> RETRYING
FAILED --> [*]
RETRYING --> [*]
[*] --> PENDING_HELD
PENDING_HELD --> FORWARDING
PENDING_HELD --> CANCELING
PENDING_HELD --> FAILED
FORWARDING --> FORWARDED
FORWARDING --> FAILED_FORWARD
FORWARDED --> PAID
FAILED_FORWARD --> CANCELING
FAILED_FORWARD --> FAILED
```
</details>
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
## Paid Action Interface
Each paid action is implemented in its own file in the `paidAction` directory. Each file exports a module with the following properties:
### Boolean flags
- `anonable`: can be performed anonymously
- `supportsPessimism`: supports a pessimistic payment flow
- `supportsOptimism`: supports an optimistic payment flow
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
### Functions
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
All functions have the following signature: `function(args: Object, context: Object): Promise`
- `getCost`: returns the cost of the action in msats as a `BigInt`
- `perform`: performs the action
- returns: an object with the result of the action as defined in the `graphql` schema
- if the action supports optimism and an `invoiceId` is provided, the action should be performed optimistically
- any action data that needs to be hidden while it's pending, should store in its rows a `PENDING` state along with its `invoiceId`
- it can optionally store in the invoice with the `invoiceId` the `actionId` to be able to link the action with the invoice regardless of retries
- `onPaid`: called when the action is paid
- if the action does not support optimism, this function is optional
- this function should be used to mark the rows created in `perform` as `PAID` and perform any other side effects of the action (like notifications or denormalizations)
- `onFail`: called when the action fails
- if the action does not support optimism, this function is optional
- this function should be used to mark the rows created in `perform` as `FAILED`
- `retry`: called when the action is retried with any new invoice information
- return: an object with the result of the action as defined in the `graphql` schema (same as `perform`)
- this function is called when an optimistic action is retried
- it's passed the original `invoiceId` and the `newInvoiceId`
- this function should update the rows created in `perform` to contain the new `newInvoiceId` and remark the row as `PENDING`
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
- `invoiceablePeer`: returns the userId of the peer that's capable of generating an invoice so they can be paid for the action
- this is only used for p2p wrapped zaps currently
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
- `describe`: returns a description as a string of the action
- for actions that require generating an invoice, and for stackers that don't hide invoice descriptions, this is used in the invoice description
#### Function arguments
`args` contains the arguments for the action as defined in the `graphql` schema. If the action is optimistic or pessimistic, `args` will contain an `invoiceId` field which can be stored alongside the paid action's data. If this is a call to `retry`, `args` will contain the original `invoiceId` and `newInvoiceId` fields.
`context` contains the following fields:
- `me`: the user performing the action (undefined if anonymous)
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
- `cost`: the cost of the action in msats as a `BigInt`
- `tx`: the current transaction (for anything that needs to be done atomically with the payment)
- `models`: the current prisma client (for anything that doesn't need to be done atomically with the payment)
- `lnd`: the current lnd client
## `IMPORTANT: transaction isolation`
We use a `read committed` isolation level for actions. This means paid actions need to be mindful of concurrency issues. Specifically, reading data from the database and then writing it back in `read committed` is a common source of consistency bugs (aka serialization anamolies).
### This is a big deal
1. If you read from the database and intend to use that data to write to the database, and it's possible that a concurrent transaction could change the data you've read (it usually is), you need to be prepared to handle that.
2. This applies to **ALL**, and I really mean **ALL**, read data regardless of how you read the data within the `read committed` transaction:
- independent statements
- `WITH` queries (CTEs) in the same statement
- subqueries in the same statement
### How to handle it
1. take row level locks on the rows you read, using something like a `SELECT ... FOR UPDATE` statement
- NOTE: this does not protect against missing concurrent inserts. It only prevents concurrent updates to the rows you've already read.
- read about row level locks available in postgres: https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-ROWS
2. check that the data you read is still valid before writing it back to the database i.e. optimistic concurrency control
- NOTE: this does not protect against missing concurrent inserts. It only prevents concurrent updates to the rows you've already read.
3. avoid having to read data from one row to modify the data of another row all together
### Example
Let's say you are aggregating total sats for an item from a table `zaps` and updating the total sats for that item in another table `item_zaps`. Two 100 sat zaps are requested for the same item at the same time in two concurrent transactions. The total sats for the item should be 200, but because of the way `read committed` works, the following statements lead to a total sats of 100:
*the statements here are listed in the order they are executed, but each transaction is happening concurrently*
#### Incorrect
```sql
-- transaction 1
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
SELECT sum(sats) INTO total_sats FROM zaps WHERE item_id = 1;
-- total_sats is 100
-- transaction 2
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
SELECT sum(sats) INTO total_sats FROM zaps WHERE item_id = 1;
-- total_sats is still 100, because transaction 1 hasn't committed yet
-- transaction 1
UPDATE item_zaps SET sats = total_sats WHERE item_id = 1;
-- sets sats to 100
-- transaction 2
UPDATE item_zaps SET sats = total_sats WHERE item_id = 1;
-- sets sats to 100
COMMIT;
-- transaction 1
COMMIT;
-- item_zaps.sats is 100, but we would expect it to be 200
```
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
Note that row level locks wouldn't help in this case, because we can't lock the rows that the transactions don't know to exist yet.
backend payment optimism (#1195) * wip backend optimism * another inch * make action state transitions only happen once * another inch * almost ready for testing * use interactive txs * another inch * ready for basic testing * lint fix * inches * wip item update * get item update to work * donate and downzap * inchy inch * fix territory paid actions * wip usePaidMutation * usePaidMutation error handling * PENDING_HELD and HELD transitions, gql paidAction return types * mostly working pessimism * make sure invoice field is present in optimisticResponse * inches * show optimistic values to current me * first pass at notifications and payment status reporting * fix migration to have withdrawal hash * reverse optimism on payment failure * Revert "Optimistic updates via pending sats in item context (#1229)" This reverts commit 93713b33df9bc3701dc5a692b86a04ff64e8cfb1. * add onCompleted to usePaidMutation * onPaid and onPayError for new comments * use 'IS DISTINCT FROM' for NULL invoiceActionState columns * make usePaidMutation easier to read * enhance invoice qr * prevent actions on unpaid items * allow navigation to action's invoice * retry create item * start edit window after item is paid for * fix ux of retries from notifications * refine retries * fix optimistic downzaps * remember item updates can't be retried * store reference to action item in invoice * remove invoice modal layout shift * fix destructuring * fix zap undos * make sure ItemAct is paid in aggregate queries * dont toast on long press zap undo * fix delete and remindme bots * optimistic poll votes with retries * fix retry notifications and invoice item context * fix pessimisitic typo * item mentions and mention notifications * dont show payment retry on item popover * make bios work * refactor paidAction transitions * remove stray console.log * restore docker compose nwc settings * add new todos * persist qr modal on post submission + unify item form submission * fix post edit threshold * make bounty payments work * make job posting work * remove more store procedure usage ... document serialization concerns * dont use dynamic imports for paid action modules * inline comment denormalization * create item starts with median votes * fix potential of serialization anomalies in zaps * dont trigger notification indicator on successful paid action invoices * ignore invoiceId on territory actions and add optimistic concurrency control * begin docs for paid actions * better error toasts and fix apollo cache warnings * small documentation enhancements * improve paid action docs * optimistic concurrency control for territory updates * use satsToMsats and msatsToSats helpers * explictly type raw query template parameters * improve consistency of nested relation names * complete paid action docs * useEffect for canEdit on payment * make sure invoiceId is provided when required * don't return null when expecting array * remove buy credits * move verifyPayment to paidAction * fix comments invoicePaidAt time zone * close nwc connections once * grouped logs for paid actions * stop invoiceWaitUntilPaid if not attempting to pay * allow actionState to transition directly from HELD to PAID * make paid mutation wait until pessimistic are fully paid * change button text when form submits/pays * pulsing form submit button * ignore me in notification indicator for territory subscription * filter unpaid items from more queries * fix donation stike timing * fix pending poll vote * fix recent item notifcation padding * no default form submitting button text * don't show paying on submit button on free edits * fix territory autorenew with fee credits * reorg readme * allow jobs to be editted forever * fix image uploads * more filter fixes for aggregate views * finalize paid action invoice expirations * remove unnecessary async * keep clientside cache normal/consistent * add more detail to paid action doc * improve paid action table * remove actionType guard * fix top territories * typo api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * typo components/use-paid-mutation.js Co-authored-by: ekzyis <ek@stacker.news> * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> * encorporate ek feeback * more ek suggestions * fix 'cost to post' hover on items * Apply suggestions from code review Co-authored-by: ekzyis <ek@stacker.news> --------- Co-authored-by: ekzyis <ek@stacker.news>
2024-07-01 17:02:29 +00:00
#### Subqueries are still incorrect
```sql
-- transaction 1
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
UPDATE item_zaps SET sats = (SELECT sum(sats) INTO total_sats FROM zaps WHERE item_id = 1) WHERE item_id = 1;
-- item_zaps.sats is 100
-- transaction 2
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
UPDATE item_zaps SET sats = (SELECT sum(sats) INTO total_sats FROM zaps WHERE item_id = 1) WHERE item_id = 1;
-- item_zaps.sats is still 100, because transaction 1 hasn't committed yet
-- transaction 1
COMMIT;
-- transaction 2
COMMIT;
-- item_zaps.sats is 100, but we would expect it to be 200
```
Note that while the `UPDATE` transaction 2's update statement will block until transaction 1 commits, the subquery is computed before it blocks and is not re-evaluated after the block.
#### Correct
```sql
-- transaction 1
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
-- transaction 2
BEGIN;
INSERT INTO zaps (item_id, sats) VALUES (1, 100);
-- transaction 1
UPDATE item_zaps SET sats = sats + 100 WHERE item_id = 1;
-- transaction 2
UPDATE item_zaps SET sats = sats + 100 WHERE item_id = 1;
COMMIT;
-- transaction 1
COMMIT;
-- item_zaps.sats is 200
```
The above works because `UPDATE` takes a lock on the rows it's updating, so transaction 2 will block until transaction 1 commits, and once transaction 2 is unblocked, it will re-evaluate the `sats` value of the row it's updating.
#### More resources
- https://stackoverflow.com/questions/61781595/postgres-read-commited-doesnt-re-read-updated-row?noredirect=1#comment109279507_61781595
- https://www.cybertec-postgresql.com/en/transaction-anomalies-with-select-for-update/
From the [postgres docs](https://www.postgresql.org/docs/current/transaction-iso.html#XACT-READ-COMMITTED):
> UPDATE, DELETE, SELECT FOR UPDATE, and SELECT FOR SHARE commands behave the same as SELECT in terms of searching for target rows: they will only find target rows that were committed as of the command start time. However, such a target row might have already been updated (or deleted or locked) by another concurrent transaction by the time it is found. In this case, the would-be updater will wait for the first updating transaction to commit or roll back (if it is still in progress). If the first updater rolls back, then its effects are negated and the second updater can proceed with updating the originally found row. If the first updater commits, the second updater will ignore the row if the first updater deleted it, otherwise it will attempt to apply its operation to the updated version of the row. The search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches the search condition. If so, the second updater proceeds with its operation using the updated version of the row. In the case of SELECT FOR UPDATE and SELECT FOR SHARE, this means it is the updated version of the row that is locked and returned to the client.
From the [postgres source docs](https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/executor/README#l350):
2024-07-26 01:45:54 +00:00
> It is also possible that there are relations in the query that are not to be locked (they are neither the UPDATE/DELETE/MERGE target nor specified to be locked in SELECT FOR UPDATE/SHARE). When re-running the test query ***we want to use the same rows*** from these relations that were joined to the locked rows.
## `IMPORTANT: deadlocks`
Deadlocks can occur when two transactions are waiting for each other to release locks. This can happen when two transactions lock rows in different orders whether explicit or implicit.
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
If both transactions lock the rows in the same order, the deadlock is avoided.
2024-07-26 01:45:54 +00:00
### Incorrect
```sql
-- transaction 1
BEGIN;
UPDATE users set msats = msats + 1 WHERE id = 1;
-- transaction 2
BEGIN;
UPDATE users set msats = msats + 1 WHERE id = 2;
-- transaction 1 (blocks here until transaction 2 commits)
UPDATE users set msats = msats + 1 WHERE id = 2;
-- transaction 2 (blocks here until transaction 1 commits)
UPDATE users set msats = msats + 1 WHERE id = 1;
-- deadlock occurs because neither transaction can proceed to here
```
not-custodial zap beta (#1178) * not-custodial zap scaffolding * invoice forward state machine * small refinements to state machine * make wrap invoice work * get state machine working end to end * untested logic layout for paidAction invoice wraps * perform pessimisitic actions before outgoing payment * working end to end * remove unneeded params from wallets/server/createInvoice * fix cltv relative/absolute confusion + cancelling forwards * small refinements * add p2p wrap info to paidAction docs * fallback to SN invoice when wrap fails * fix paidAction retry description * consistent naming scheme for state machine * refinements * have sn pay bounded outbound fee * remove debug logging * reenable lnc permissions checks * don't p2p zap on item forward splits * make createInvoice params json encodeable * direct -> p2p badge on notifications * allow no tls in dev for core lightning * fix autowithdraw to create invoice with msats * fix autowithdraw msats/sats inconsitency * label p2p zaps properly in satistics * add fees to autowithdrawal notifications * add RETRYING as terminal paid action state * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/paidAction/README.md Co-authored-by: ekzyis <ek@stacker.news> * Update api/lnd/index.js Co-authored-by: ekzyis <ek@stacker.news> * ek suggestions * add bugetable to nwc card * get paranoid with numbers * better finalize retries and better max timeout height * refine forward failure transitions * more accurate satistics p2p status * make sure paidaction cancel in state machine only * dont drop bolt11s unless status is not null * only allow PENDING_HELD to transition to FORWARDING * add mermaid state machine diagrams to paid action doc * fix cancel transition name * cleanup readme * move forwarding outside of transition * refine testServerConnect and make sure ensureB64 transforms * remove unused params from testServerConnect --------- Co-authored-by: ekzyis <ek@stacker.news> Co-authored-by: k00b <k00b@stacker.news>
2024-08-13 14:48:30 +00:00
In practice, this most often occurs when selecting multiple rows for update in different orders. Recently, we had a deadlock when spliting zaps to multiple users. The solution was to select the rows for update in the same order.
2024-07-26 01:45:54 +00:00
### Incorrect
```sql
WITH forwardees AS (
SELECT "userId", (($1::BIGINT * pct) / 100)::BIGINT AS msats
FROM "ItemForward"
WHERE "itemId" = $2::INTEGER
),
UPDATE users
SET
msats = users.msats + forwardees.msats,
"stackedMsats" = users."stackedMsats" + forwardees.msats
FROM forwardees
WHERE users.id = forwardees."userId";
```
If forwardees are selected in a different order in two concurrent transactions, e.g. (1,2) in tx 1 and (2,1) in tx 2, a deadlock can occur. To avoid this, always select rows for update in the same order.
### Correct
We fixed the deadlock by selecting the forwardees in the same order in these transactions.
```sql
WITH forwardees AS (
SELECT "userId", (($1::BIGINT * pct) / 100)::BIGINT AS msats
FROM "ItemForward"
WHERE "itemId" = $2::INTEGER
ORDER BY "userId" ASC
),
UPDATE users
SET
msats = users.msats + forwardees.msats,
"stackedMsats" = users."stackedMsats" + forwardees.msats
FROM forwardees
WHERE users.id = forwardees."userId";
```
### More resources
- https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-DEADLOCKS