371e7417ce
* wip: Use uniform interface for wallets * Fix import error * Update wallet logging + other stuff * add canPay and canSend to wallet definition * rename 'default payment method' to 'enabled' and add enable + disable method * Set canPay, canReceive in useWallet * Enable wallet if just configured * Don't pass logger to sendPayment * Add logging to attach & detach * Add schema to wallet def * Add NWC wallet * Fix unused isDefault saved in config * Fix enableWallet * wrong storage key was used * broke if wallets with no configs existed * Run validation during save * Use INFO level for 'wallet disabled' message * Pass config with spread operator * Support help, optional, hint in wallet fields * wip: Add LNC * Fix 20s page load for /settings/wallets.json?nodata=true For some reason, if nodata is passed (which is the case if going back), the page takes 20s to load. * Fix extremely slow page load for LNC import I noticed that the combination of ``` import { Form, PasswordInput, SubmitButton } from '@/components/form' ``` in components/wallet/lnc.js and the dynamic import via `await import` in components/wallet/index.js caused extremely slow page loads. * Use normal imports * Revert "Fix 20s page load for /settings/wallets.json?nodata=true" This reverts commit deb476b3a966569fefcfdf4082d6b64f90fbd0a2. Not using the dynamic import for LNC fixed the slow page load with ?nodata=true. * Remove follow and show recent logs first * Fix position of log start marker * Add FIXMEs for LNC I can't get LNC to connect. It just hangs forever on lnc.connect(). See FIXMEs. * Remove logger.error since already handled in useWallet * Don't require destructuring to pass props to input * wip: Add LND autowithdrawals * receiving wallets need to export 'server' object field * don't print macaroon error stack * fix missing wallet logs order update * mark autowithdrawl settings as required * fix server wallet logs deletion * remove canPay and canReceive since it was confusing where it is available TODO * also use numeric priority for sending wallets to be consistent with how status for receiving wallets is determined * define createInvoice function in wallet definition * consistent wallet logs: sending wallets use 'wallet attached'+'wallet enabled/disabled' whereas receiving wallets use 'wallet created/updated' * see FIXMEs * Fix TypeError * Fix sendPayment called with empty config * removed useEffect such that config is available on first render * fix hydration error using dynamic import without SSR * Fix confusing UX around enabled * Remove FIXMEs Rebase on master seemed to have fixed these, weird * Use same error format in toast and wallet log * Fix usage of conditional hooks in useConfig * Fix isConfigured * Fix delete wallet logs on server * Fix wallet logs refetch onError does not exist on client.mutate * Fix TypeError in isConfigured if no enabled wallet found * Only include local/server config if required * Fix another hydration error * Fix server config not updated after save or detach * Also use 'enabled' for server wallets * Fix wallet logs not updated after server delete * Consistent logs between local and server wallets * 'wallet attached' on create * 'wallet updated' on config updates * 'wallet enabled' and 'wallet disabled' if checkbox changed * 'wallet detached' on delete * Also enable server wallets on create * Disable checkbox if not configured yet * Move all validation schema into lib/validate * Implement drag & drop w/o persistence * Use dynamic import for WalletCard This fixes a lot of issues with hydration * Save order as priority * Fix autowithdrawSettings not applied Form requires config in flat format but mutation requires autowithdraw settings in a separate 'settings' field. I have decided that config will be in flat form format. It will be transformed into mutation format during save. * Save dedicated enabled flag for server wallets * wallet table now contains boolean column 'enabled' * 'priority' is now a number everywhere * use consistent order between how autowithdrawals are attempted and server wallets cards * Fix onCanceled missing * Fix typo * Fix noisy changes in lib/validate I moved the schema for lnbits, nwc and lnc out of lib/validate only to put them back in there later. This commit should make the changeset cleaner by removing noise. * Split arguments into [value,] config, context * Run lnbits url.replace in validate and sendPayment * Remove unnecessary WALLETS_QUERY * Generate wallet mutation from fields * Generate wallet resolver from fields * Fix import inconsistency between app and worker * Use wallet.createInvoice for autowithdrawals * Fix success autowithdrawal log * Fix wallet security banner shown for server wallets * Add autowithdrawal to lightning address * Add optional wallet short name for logging * Fix draggable * Fix autowithdraw loop * Add missing hints * Add CLN autowithdrawal * Detach wallets and delete logs on logout * Remove Wallet in lib/constants * Use inject function for resolvers and typeDefs * Fix priority ignored when fetching enabled wallet * Fix draggable false on first page load due to SSR * Use touches instead of dnd on mobile Browsers don't support drag events for touch devices. To have a consistent implementation for desktop and mobile, we would need to use mousedown/touchstart, mouseup/touchend and mousemove/touchmove. For now, this commit makes changing the order possible on touch devices with simple touches. * Fix duplicate CLN error * Fix autowithdraw priority order * Fix error per invalid bip39 word * Update LNC code * remove LNC FIXMEs Mhh, I guess the TURN server was down or something? It now magically works. Or maybe it only works once per mnemonic? * also removed the lnc.lnd.lightning.getInfo() call since we don't ask and need permission for this RPC for payments. * setting a password does not work though. It fails with 'The password provided is not valid' which is triggered at https://github.com/lightninglabs/lnc-web/blob/main/lib/util/credentialStore.ts#L81. * Fix order if wallet with no priority exists * Use common sort * Add link to lnbits.com * Add example wallet def * Remove TODOs TODO in components/wallet-logger.js was handled. I don't see a need for the TODO in lib/wallet.js anymore. This function will only be called with the wallet of type LIGHTNING_ADDRESS anyway. * Remove console.log * Toast priority save errors * Fix leaking relay connections * Remove 'tor or clearnet' hint for LN addresses * Remove React dependency from wallet definitions * Generate resolver name from walletField * Move wallets into top level directory wallet/ * Put wallets into own folder * Fix generateMutation * remove resolverName property from wallet defs * move function into lib/wallet * use function in generateMutation on client to fix wrongly generated mutation * Separate client and server imports by files * wallets now consist of an index.js, a client.js and a server.js file * client.js is imported on the client and contains the client portion * server.js is imported on the server and contains the server porition * both reexport index.js so everything in index.js can be shared by client and server * every wallet contains a client.js file since they are all imported on the client to show the cards * client.js of every wallet is reexported as an array in wallets/client.js * server.js of every wallet is reexported as an array in wallets/server.js FIXME: for some reason, worker does not properly import the default export of wallets/server.js * Fix worker import of wallets/server * Fix wallet.server usage * I removed wallet.server in a previous commit * the client couldn't determine which wallet was stored on the server since all server specific fields were set in server.js * walletType and walletField are now set in index.js * walletType is now used to determine if a wallet is stored on the server * also included some formatting changes * Fix w.default usage Since package.json with { "type": "module" } was added, this is no longer needed. * Fix id access in walletPrioritySort * Fix autowithdrawal error log * Generate validation schema for LNbits * Generate validation schema for NWC * Rename to torAllowed * Generate validation schema for LNC * Generate validation schema for LND * Generate validation schema for LnAddr * Remove stringTypes * Generate validation schema for CLN * Make clear that message belongs to test * validate.message was used in tandem with validate.test * it might be confused as the message if the validation for validate.type failed * now validate.test can be a function or an object of { test, message } shape which matches Yup.test * Remove validate.schema as a trap door * make lnc work * Return null if no wallet was found * Revert code around schema generation * Transform autowithdrawSchemaMembers into an object * Rename schema to yupSchema * Fix missing required for LNbits adminKey * Support formik form-level validation * Fix missing addWalletLog import * Fix missing space after = * fix merge conflict resolution mistake * remove non-custodial* badges * create guides for attaching wallets in sndev * Use built-in formik validation or Yup schema but not both * Rename: validate -> testConnectClient, testConnect -> testConnectServer * make lnaddr autowithdraw work in dev * move ATTACH docs to ./wallets and add lnaddr doc * Fix missing rename: yupSchema -> fieldValidation * Remove unused context * Add documentation how to add wallets --------- Co-authored-by: keyan <keyan.kousha+huumn@gmail.com> Co-authored-by: Keyan <34140557+huumn@users.noreply.github.com> |
||
---|---|---|
.ebextensions | ||
.github | ||
.platform | ||
.vscode | ||
api | ||
capture | ||
chat-web | ||
components | ||
copilot | ||
docker | ||
docs | ||
fragments | ||
lib | ||
pages | ||
prisma | ||
public | ||
scripts | ||
styles | ||
svgs | ||
sw | ||
wallets | ||
worker | ||
.env.development | ||
.env.production | ||
.gitattributes | ||
.gitignore | ||
.npmrc | ||
Dockerfile | ||
LICENSE | ||
Procfile | ||
README.md | ||
awards.csv | ||
contributors.txt | ||
docker-compose.yml | ||
jsconfig.json | ||
middleware.js | ||
next.config.js | ||
package-lock.json | ||
package.json | ||
sndev |
README.md
- Stacker News makes internet communities that pay you Bitcoin
- What You See is What We Ship (look ma, I invented an initialism)
- 100% FOSS
- We pay bitcoin for PRs, issues, documentation, code reviews and more
- Next.js, postgres, graphql, and lnd
Getting started
Launch a fully featured SN development environment in a single command.
$ ./sndev start
Go to localhost:3000.
Installation
- Clone the repo
- ssh:
git clone git@github.com:stackernews/stacker.news.git
- https:
git clone https://github.com/stackernews/stacker.news.git
- ssh:
- Install docker
Usage
Start the development environment
$ ./sndev start
View all available commands
$ ./sndev help
888
888
888
.d8888b 88888b. .d88888 .d88b. 888 888
88K 888 '88b d88' 888 d8P Y8b 888 888
'Y8888b. 888 888 888 888 88888888 Y88 88P
X88 888 888 Y88b 888 Y8b. Y8bd8P
88888P' 888 888 'Y88888 'Y8888 Y88P
manages a docker based stacker news development environment
USAGE
$ sndev [COMMAND]
$ sndev help [COMMAND]
COMMANDS
help show help
env:
start start env
stop stop env
restart restart env
status status of env
logs logs from env
delete delete env
sn:
login login as a nym
lnd:
fund pay a bolt11 for funding
withdraw create a bolt11 for withdrawal
cln:
cln_fund pay a bolt11 for funding with CLN
cln_withdraw create a bolt11 for withdrawal with CLN
db:
psql open psql on db
prisma run prisma commands
dev:
pr fetch and checkout a pr
lint run linters
other:
compose docker compose passthrough
sn_lndcli lncli passthrough on sn_lnd
stacker_lndcli lncli passthrough on stacker_lnd
stacker_clncli lightning-cli passthrough on stacker_cln
Modifying services
Running specific services
By default all services will be run. If you want to exclude specific services from running, set COMPOSE_PROFILES
to use one or more of minimal|images|search|payments|wallets|email|capture
. To only run mininal services without images, search, email, wallets, or payments:
$ COMPOSE_PROFILES=minimal ./sndev start
Or, as I would recommend:
$ export COMPOSE_PROFILES=minimal
$ ./sndev start
To run with images and payments services:
$ COMPOSE_PROFILES=images,payments ./sndev start
Merging compose files
By default sndev start
will merge docker-compose.yml
with docker-compose.override.yml
. Specify any overrides you want to merge with docker-compose.override.yml
.
For example, if you want to replace the db seed with a custom seed file located in docker/db/another.sql
, you'd create a docker-compose.override.yml
file with the following:
services:
db:
volumes:
- ./docker/db/another.sql:/docker-entrypoint-initdb.d/seed.sql
You can read more about docker compose override files.
Table of Contents
- Getting started
- Contributing
- Contribution extras
- Development Tips
- Internals
- Need help?
- Responsible Disclosure
- License
Contributing
We want your help.
We pay bitcoin for contributions
- pull requests closing existing issues
- code review
- issue specification whether for bugs, features, or enhancements
- discovery of security vulnerabilities
- discovery of privacy vulnerabilities
- improvements to development documentation
- helpfulness
View a current list of granted awards
Just in case
This document in no way legally entitles you to payments for contributions, entitles you to being a contributor, or entitles you to the attention of other contributors. This document lays out the system we can use to determine contribution awards which we generally intend to abide by but importantly we reserve the right to refuse payments or contributions, modify rules and award amounts, make exceptions to rules or reward amounts, and withhold awards for any reason at anytime, even just for the heck of it, at our sole discretion. If you need more certainty than what I've just described, don't participate. We provide awards as an experiment to make FOSS less sucky.
Pull request awards
Rules
- PRs closing an issue will be awarded according to the
difficulty
tag on an issue, e.g.difficulty:easy
pays 100k sats. - Issues are occasionally marked with a
priority
tag which multiplies the award of a PR closing an issue, e.g. an issue marked withpriority:high
anddifficulty:hard
awards 2m sats. - An award is reduced by 10% of the award amount for each substantial change requested to the PR on code review, e.g. if two changes are requested on a PR closing an issue tagged with
difficulty:hard
, 800k sats will be awarded.- Reductions are applied before
priority
multipliers, e.g. a PR closing apriority:high
anddifficulty:hard
issue that's approved after two changes are requested awards 1.6m sats. - You are responsible for understanding the issue and requirements before requesting review on a PR.
- There is no award reduction for asking specific questions on the issue itself or on the PR before requesting review
- There is no award reduction for asking more general questions in a discussion
- Reductions are applied before
- A PR must be merged by an SN engineer before a PR receives an award
Due to Rule 3, make sure that you mark your PR as a draft when you create it and it's not ready for review yet.
Difficulty award amounts
tag | description | award |
---|---|---|
difficulty:good-first-issue |
at most a couple lines of code in a couple files and does not require much familiarity with the codebase | 20k sats |
difficulty:easy |
at most a couple lines of code in a couple files but does require familiarity with the code base | 100k sats |
difficulty:medium |
more code, more places and could require adding columns in the db and some modification chunky db queries | 250k sats |
difficulty:medium-hard |
even more code, even more places and requires either more sophisticated logic, more significant db modeling eg adding a table, and/or a deeper study of a something | 500k sats |
difficulty:hard |
either a bigger lift than the what's required of medium-hard or very tricky in a particular way that might not require a lot of code but does require a lot of context/troubleshooting/expertise | 1m sats |
Priority multipliers
tag | multiplier |
---|---|
priority:medium |
1.5 |
priority:high |
2 |
priority:urgent |
3 |
Requesting modifications to reward amounts
We try to assign difficulty and priority tags to issues accurately, but we're not perfect. If you believe an issue is mis-tagged, you can request a change to the issue's tags.
Code review awards
Code reviewers will be awarded the amount their code review reduced from the PR author's reward, e.g. two substantial problems/areas of improvement identified in a PR closing a priority:high
and difficulty:hard
issue awards 400k sats.
Rules
- The problem or improvement must be acknowledged as such by SN engineers explicitly
- A PR must be merged by an SN engineer before a PR's code reviewers receive an award
Code review approvals are more than welcome, but we can't guarantee awards for them because the work performed to approve a PR is unverifiable.
Issue specification awards
Issue specifiers will be awarded up to 10% of a PR award for issues resulting in a PR being merged by an SN engineer that closes the issue. In addition to being subject to PR award amounts and reductions, specification amounts are awarded on the basis of how much additional help and specification is required by other contributors.
Rules
- The issue must directly result in PR being merged by an SN engineer that closes the issue
- Issue specification award amounts are based on the final PR award amounts
- that is, they are subject to PR award code review reductions and priority multipliers
- Award amounts will be reduced on the basis of how much additional help and specification is required by other contributors
- Issue specifiers who can close their own issues with their own PRs are also eligible for this 10%
- e.g an issue tagged as
difficulty:hard
that is both specified and closed by a PR from the same contributor without changes requested awards 1.1m sats
- e.g an issue tagged as
Relative awards
circumstances | award |
---|---|
issue doesn't require further help and/or specification from other contributors | 10% |
issue requires little help and/or specification from other contributors | 5% |
issue requires more help and/or specification from other contributors than the issue specifier contributed | 1% |
issue is vague and/or incomplete and must mostly be entirely specified by someone else | 0% |
For example: a specified issue that's tagged as difficulty:hard
, doesn't require additional specification and disambiguation by other contributors, and results in PR being merged without changes requested awards the issue specifier 100k sats.
Responsible disclosure of security or privacy vulnerability awards
Awards for responsible disclosures are assessed on the basis of:
- the potential loss resulting from an exploit of the vulnerability
- the trivialness of exploiting the vulnerability
- the disclosure's detail
Award amounts will be easiest to assess on a case by case basis. Upon confirmation of a vulnerability, we agree to award responsible disclosures at minimum 100k sats and as high as the total potential loss that would result from exploiting the vulnerability.
Rules
- Disclosure is responsible and does not increase the likelihood of an exploit.
- Disclosure includes steps to reproduce.
- Disclosure includes a realistic attack scenario with prerequisites for an attack and expected gains after the exploitation. Disclosures without such scenario, with unrealistic assumptions or without meaningful outcomes will not be eligible for awards.
- You must be the first person to responsibly disclose the issue to be eligible for awards.
Development documentation awards
For significant changes to documentation, create an issue before making said changes. In such cases we will award documentation improvements in accordance with issue specification and PR awards.
For changes on the order of something like a typo, we'll award a nominal amount at our discretion.
Helpfulness awards
Like issue specification awards, helping fellow contributors substantially in a well documented manner such that the helped fellow contributes a merged PR is eligible for a one-time relative reward.
circumstances | award |
---|---|
substantial and singular source of help | 10% |
substantial but nonsingular source of help | 1-5% |
source of relatively trivial help | 1% |
Contribution extras
We want to make contributing to SN as rewarding as possible, so we offer a few extras to contributors.
Dev chat
We self-host a private chat server for contributors to SN. If you'd like to join, please respond in this discussion.
Triage permissions
We offer triage permissions to contributors after they've made a few contributions. I'll usually add them as I notice people contributing, but if I missed you and you'd like to be added, let me know!
Contributor badges on SN profiles
Contributors can get badges on their SN profiles by opening a pull request adding their SN nym to the contributors.txt file.
What else you got
In the future we plan to offer more, like gratis github copilot subscriptions, reverse tunnels, codespaces, and merch.
If you'd like to see something added, please make a suggestion.
Development Tips
Linting
We use JavaScript Standard Style to enforce code style and correctness. You should run sndev lint
before submitting a PR.
If you're using VSCode, you can install the StandardJS VSCode Extension extension to get linting in your editor. We also recommend installing StandardJS code snippets and StandardJS react code snippets for code snippets.
Database migrations
We use prisma for our database migrations. To create a new migration, modify prisma/schema.prisma
according to prisma schema reference and apply it with:
./sndev prisma migrate dev
If you want to create a migration without applying it, eg to create a trigger or modify the generated sql before applying, use the --create-only
option:
./sndev prisma migrate dev --create-only
Generate the local copy of the prisma ORM client in node_modules
after changes. This should only be needed to get Intellisense in your editor locally.
./sndev prisma generate
Connecting to the local database
You can connect to the local database via ./sndev psql
. psql is an interactive terminal for working with PostgreSQL.
Running lncli on the local lnd nodes
You can run lncli
on the local lnd nodes via ./sndev sn_lncli
and ./sndev stacker_lncli
. The node for your local SN instance is sn_lnd
and the node serving as any external node, like a stacker's node or external wallet, is stacker_lnd
.
Testing local auth
You can login to test features like posting, replying, tipping, etc with ./sndev login <nym>
which will provide a link to login as an existing nym or a new account for a nonexistent nym. But, it you want to test auth specifically you'll need to configure them in your .env
file.
Login with Email
MailHog
- The app is already prepared to send emails through MailHog so no extra configuration is needed
- Click "sign up" and enter any email address (remember, it's not going anywhere beyond your workstation)
- Access MailHog's web UI on http://localhost:8025
- Click the link (looks like this):
http://localhost:3000/api/auth/callback/email?email=satoshi%40gmail.com&token=110e30a954ce7ca643379d90eb511640733de405f34a31b38eeda8e254d48cd7
Sendgrid
- Create a Sendgrid account (or other smtp service)
LOGIN_EMAIL_SERVER=smtp://apikey:<sendgrid_api_key>@smtp.sendgrid.net:587
LOGIN_EMAIL_FROM=<sendgrid_email_from>
- Click "sign up" and enter your email address
- Check your email
- Click the link (looks like this):
http://localhost:3000/api/auth/callback/email?email=satoshi%40gmail.com&token=110e30a954ce7ca643379d90eb511640733de405f34a31b38eeda8e254d48cd7
Login with Github
- Create a new OAuth app in your Github account
- Set the callback URL to:
http://localhost:3000/api/auth/callback/github
- Set the callback URL to:
- Update your
.env
file
GITHUB_ID=<Client ID>
GITHUB_SECRET=<Client secret>
- Signup and login as above
Login with Lightning
- Use ngrok to create a HTTPS tunnel to localhost:3000
- Update
LNAUTH_URL
in.env
with the URL provided byngrok
and add /api/lnauth to it
Enabling web push notifications
To enable Web Push locally, you will need to set the VAPID_*
env vars. VAPID_MAILTO
needs to be an email address using the mailto:
scheme. For NEXT_PUBLIC_VAPID_KEY
and VAPID_PRIVKEY
, you can run npx web-push generate-vapid-keys
.
Internals
Stack
The site is written in javascript (not typescript 😱) using Next.js, a React framework. The backend API is provided via GraphQL. The database is PostgreSQL modeled with Prisma. The job queue is also maintained in PostgreSQL. We use lnd for our lightning node. A customized Bootstrap theme is used for styling.
Services
Currently, SN runs and maintains two significant services and one microservice:
- the nextjs web app, found in
./
- the worker service, found in
./worker
, which runs periodic jobs and jobs sent to it by the web app - a screenshot microservice, found in
./capture
, which takes screenshots of SN for social previews
In addition, we run other critical services the above services interact with like lnd
, postgres
, opensearch
, tor
, and s3
.
Wallet transaction safety
To ensure stackers balances are kept sane, all wallet updates are run in serializable transactions at the database level. Because early versions of prisma had relatively poor support for transactions most wallet touching code is written in plpgsql stored procedures and can be found in the prisma/migrations
folder.
Need help?
Open a discussion or issue or email us or request joining the dev chat.
Responsible disclosure
If you found a vulnerability, we would greatly appreciate it if you contact us via security@stacker.news or open a security advisory. Our PGP key can be found here (EBAF 75DA 7279 CB48).