Convert worker to ESM (#500)
* Convert worker to ESM
To use ESM for the worker, I created a package.json file in worker/ with `{ type: "module" }` as its sole content.
I then rewrote every import to use ESM syntax.
I also tried to set `{ type: "module" }` in the root package.json file to also use ESM in next.config.js.
However, this resulted in a weird problem: default imports were now getting imported as objects in this shape: `{ default: <defaultImport> }`.
Afaik, this should only be the case if you use "import * as foo from 'bar'" syntax: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#default_import
This is fixed by not using `{ type: "module" }` for some reason. However, then, next.config.js also doesn't support ESM import syntax anymore.
The documentation says that if you want to use ESM, you can use next.config.mjs: https://nextjs.org/docs/pages/api-reference/next-config-js
But I didn't want to use MJS extension since I don't have any experience with it. For example, not sure if it's good style to mix JS with MJS etc. So I kept the CJS import syntax there.
* Ignore worker/ during linting
I wasn't able to fix the following errors:
/home/runner/work/stacker.news/stacker.news/worker/auction.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/auction.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/earn.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/earn.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/index.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/index.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/nostr.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/nostr.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/ots.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/ots.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/repin.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/repin.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/search.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/search.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/streak.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/streak.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/trust.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/trust.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/views.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/views.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
/home/runner/work/stacker.news/stacker.news/worker/wallet.js:0:0: Parsing error: No Babel config file detected for /home/runner/work/stacker.news/stacker.news/worker/wallet.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files. (null)
I tried to tell babel where to find the babel config file (.babelrc), specifying the babel config in worker/package.json under "babel", using babel.config.json etc. to no avail.
However, afaict, we don't need babel for the worker since it won't run in a browser. Babel is only used to transpile code to target browsers.
But it still would be nice to lint the worker code with standard.
But we can figure this out later.
* Fix worker imports from lib/ and api/
This fixes the issue that we can't use `{ "type": "module" }` in the root package.json since it breaks the app with this error:
app | TypeError: next_auth_providers_credentials__WEBPACK_IMPORTED_MODULE_2__ is not a function
app | at eval (webpack-internal:///./pages/api/auth/[...nextauth].js:218:20)
app | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
app | LND GRPC connection successful
app | - error pages/api/auth/[...nextauth].js (139:2) @ CredentialsProvider
app | - error Error [TypeError]: next_auth_providers_credentials__WEBPACK_IMPORTED_MODULE_2__ is not a function
app | at eval (webpack-internal:///./pages/api/auth/[...nextauth].js:218:20) {
app | digest: undefined
app | }
app | 137 |
app | 138 | const providers = [
app | > 139 | CredentialsProvider({
app | | ^
app | 140 | id: 'lightning',
app | 141 | name: 'Lightning',
app | 142 | credentials: {
app | TypeError: next_auth_providers_credentials__WEBPACK_IMPORTED_MODULE_2__ is not a function
app | at eval (webpack-internal:///./pages/api/auth/[...nextauth].js:218:20)
app | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
build but we need to tell the worker that the files are MJS, else we get this error:
worker | file:///app/worker/wallet.js:3
worker | import { datePivot } from '../lib/time.js'
worker | ^^^^^^^^^
worker | SyntaxError: Named export 'datePivot' not found. The requested module '../lib/time.js' is a CommonJS module, which may not support all module.exports as named exports.
worker | CommonJS modules can always be imported via the default export, for example using:
worker |
worker | import pkg from '../lib/time.js';
worker | const { datePivot } = pkg;
worker |
worker | at ModuleJob._instantiate (node:internal/modules/esm/module_job:124:21)
worker | at async ModuleJob.run (node:internal/modules/esm/module_job:190:5)
worker |
worker | Node.js v18.17.0
worker |
worker exited with code 1
* Fix global not defined in browser context
* Also ignore api/ and lib/ during linting
I did not want to do this but I was not able to fix this error in any other way I tried:
/home/ekzyis/programming/stacker.news/api/lnd/index.js:0:0: Parsing error: No Babel config file detected for /home/ekzyis/programming/stacker.news/api/lnd/index.js. Either disable config file checking with requ
ireConfigFile: false, or configure Babel so that it can find the config files. (null)
Did not want to look deeper into all this standard, eslint, babel configuration stuff ...
---------
Co-authored-by: ekzyis <ek@stacker.news>
Co-authored-by: Keyan <34140557+huumn@users.noreply.github.com>
2023-09-24 01:19:35 +00:00
|
|
|
import serialize from '../api/resolvers/serial.js'
|
2023-10-13 19:30:58 +00:00
|
|
|
import { sendUserNotification } from '../api/webPush/index.js'
|
|
|
|
import { msatsToSats, numWithUnits } from '../lib/format.js'
|
2023-12-21 23:48:16 +00:00
|
|
|
import { PrismaClient } from '@prisma/client'
|
2024-03-01 16:28:55 +00:00
|
|
|
import { proportions } from '../lib/madness.js'
|
|
|
|
import { SN_USER_IDS } from '../lib/constants.js'
|
2022-03-17 20:13:19 +00:00
|
|
|
|
2024-03-01 16:28:55 +00:00
|
|
|
const TOTAL_UPPER_BOUND_MSATS = 10000000000
|
2022-07-07 19:14:22 +00:00
|
|
|
|
2023-12-21 23:48:16 +00:00
|
|
|
export async function earn ({ name }) {
|
2024-01-19 21:11:44 +00:00
|
|
|
// grab a greedy connection
|
2023-12-21 23:48:16 +00:00
|
|
|
const models = new PrismaClient()
|
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
try {
|
2024-03-01 16:28:55 +00:00
|
|
|
// compute how much sn earned got the month
|
2024-01-19 21:11:44 +00:00
|
|
|
const [{ sum: sumDecimal }] = await models.$queryRaw`
|
2024-03-01 16:28:55 +00:00
|
|
|
SELECT coalesce(sum(total), 0) as sum
|
|
|
|
FROM rewards_days
|
|
|
|
WHERE date_trunc('month', rewards_days.t) = date_trunc('month', (now() - interval '1 month') AT TIME ZONE 'America/Chicago')`
|
2023-08-10 02:27:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// XXX primsa will return a Decimal (https://mikemcl.github.io/decimal.js)
|
|
|
|
// because sum of a BIGINT returns a NUMERIC type (https://www.postgresql.org/docs/13/functions-aggregate.html)
|
|
|
|
// and Decimal is what prisma maps it to https://www.prisma.io/docs/concepts/components/prisma-client/raw-database-access#raw-query-type-mapping
|
|
|
|
// so check it before coercing to Number
|
|
|
|
if (!sumDecimal || sumDecimal.lessThanOrEqualTo(0)) {
|
|
|
|
console.log('done', name, 'no sats to award today')
|
|
|
|
return
|
|
|
|
}
|
2023-08-01 16:10:07 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// extra sanity check on rewards ... if it's more than upper bound, we
|
|
|
|
// probably have a bug somewhere or we've grown A LOT
|
|
|
|
if (sumDecimal.greaterThan(TOTAL_UPPER_BOUND_MSATS)) {
|
|
|
|
console.log('done', name, 'error: too many sats to award today', sumDecimal)
|
|
|
|
return
|
|
|
|
}
|
2023-08-01 16:10:07 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
const sum = Number(sumDecimal)
|
2023-08-10 02:27:53 +00:00
|
|
|
|
2024-02-22 21:31:52 +00:00
|
|
|
console.log(name, 'giving away', sum, 'msats', 'rewarding all')
|
2022-12-08 00:04:02 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
/*
|
2023-07-06 16:10:44 +00:00
|
|
|
How earnings (used to) work:
|
2022-09-12 18:55:34 +00:00
|
|
|
1/3: top 21% posts over last 36 hours, scored on a relative basis
|
|
|
|
1/3: top 21% comments over last 36 hours, scored on a relative basis
|
2022-07-07 19:14:22 +00:00
|
|
|
1/3: top upvoters of top posts/comments, scored on:
|
|
|
|
- their trust
|
|
|
|
- how much they tipped
|
|
|
|
- how early they upvoted it
|
|
|
|
- how the post/comment scored
|
2023-07-06 16:10:44 +00:00
|
|
|
|
2024-03-01 16:28:55 +00:00
|
|
|
Now: 100% of earnings go to top 33% of comments/posts and their upvoters for month
|
2022-07-07 19:14:22 +00:00
|
|
|
*/
|
2022-03-18 12:29:02 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// get earners { userId, id, type, rank, proportion }
|
|
|
|
const earners = await models.$queryRaw`
|
2024-03-01 16:28:55 +00:00
|
|
|
SELECT id AS "userId", sum(proportion) as proportion
|
|
|
|
FROM user_values_days
|
|
|
|
WHERE date_trunc('month', user_values_days.t) = date_trunc('month', (now() - interval '1 month') AT TIME ZONE 'America/Chicago')
|
|
|
|
AND NOT (id = ANY (${SN_USER_IDS}))
|
|
|
|
GROUP BY id
|
|
|
|
ORDER BY proportion DESC
|
|
|
|
LIMIT 64`
|
2022-09-12 18:55:34 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// in order to group earnings for users we use the same createdAt time for
|
|
|
|
// all earnings
|
|
|
|
const now = new Date(new Date().getTime())
|
2022-09-12 18:55:34 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// this is just a sanity check because it seems like a good idea
|
|
|
|
let total = 0
|
2022-03-18 12:29:02 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
const notifications = {}
|
2024-03-01 16:28:55 +00:00
|
|
|
for (const [i, earner] of earners.entries()) {
|
|
|
|
const earnings = Math.floor(parseFloat(proportions[i] * sum))
|
2024-01-19 21:11:44 +00:00
|
|
|
total += earnings
|
|
|
|
if (total > sum) {
|
|
|
|
console.log(name, 'total exceeds sum', total, '>', sum)
|
|
|
|
return
|
|
|
|
}
|
2022-09-12 18:55:34 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
console.log('stacker', earner.userId, 'earned', earnings, 'proportion', earner.proportion, 'rank', earner.rank, 'type', earner.type)
|
2023-08-01 16:10:07 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
if (earnings > 0) {
|
|
|
|
await serialize(models,
|
|
|
|
models.$executeRaw`SELECT earn(${earner.userId}::INTEGER, ${earnings},
|
2023-07-27 00:18:42 +00:00
|
|
|
${now}::timestamp without time zone, ${earner.type}::"EarnType", ${earner.id}::INTEGER, ${earner.rank}::INTEGER)`)
|
2023-12-21 17:35:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
const userN = notifications[earner.userId] || {}
|
2023-12-21 17:35:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// sum total
|
|
|
|
const prevMsats = userN.msats || 0
|
|
|
|
const msats = earnings + prevMsats
|
2023-12-21 17:35:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// sum total per earn type (POST, COMMENT, TIP_COMMENT, TIP_POST)
|
|
|
|
const prevEarnTypeMsats = userN[earner.type]?.msats || 0
|
|
|
|
const earnTypeMsats = earnings + prevEarnTypeMsats
|
2023-12-21 17:35:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
// best (=lowest) rank per earn type
|
|
|
|
const prevEarnTypeBestRank = userN[earner.type]?.bestRank
|
|
|
|
const earnTypeBestRank = prevEarnTypeBestRank ? Math.min(prevEarnTypeBestRank, Number(earner.rank)) : Number(earner.rank)
|
2023-12-21 17:35:53 +00:00
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
notifications[earner.userId] = {
|
|
|
|
...userN,
|
|
|
|
msats,
|
|
|
|
[earner.type]: { msats: earnTypeMsats, bestRank: earnTypeBestRank }
|
|
|
|
}
|
2022-03-18 12:29:02 +00:00
|
|
|
}
|
2023-10-13 19:30:58 +00:00
|
|
|
}
|
|
|
|
|
2024-01-19 21:11:44 +00:00
|
|
|
Promise.allSettled(Object.entries(notifications).map(([userId, earnings]) =>
|
|
|
|
sendUserNotification(parseInt(userId, 10), buildUserNotification(earnings))
|
|
|
|
)).catch(console.error)
|
|
|
|
} finally {
|
|
|
|
models.$disconnect().catch(console.error)
|
|
|
|
}
|
2023-11-21 23:32:22 +00:00
|
|
|
}
|
|
|
|
|
2023-10-13 19:30:58 +00:00
|
|
|
function buildUserNotification (earnings) {
|
|
|
|
const fmt = msats => numWithUnits(msatsToSats(msats, { abbreviate: false }))
|
|
|
|
|
2023-12-21 17:35:53 +00:00
|
|
|
const title = `you stacked ${fmt(earnings.msats)} in rewards`
|
2023-10-13 19:30:58 +00:00
|
|
|
const tag = 'EARN'
|
|
|
|
let body = ''
|
2023-12-21 17:35:53 +00:00
|
|
|
if (earnings.POST) body += `#${earnings.POST.bestRank} among posts with ${fmt(earnings.POST.msats)} in total\n`
|
|
|
|
if (earnings.COMMENT) body += `#${earnings.COMMENT.bestRank} among comments with ${fmt(earnings.COMMENT.msats)} in total\n`
|
|
|
|
if (earnings.TIP_POST) body += `#${earnings.TIP_POST.bestRank} in post zapping with ${fmt(earnings.TIP_POST.msats)} in total\n`
|
|
|
|
if (earnings.TIP_COMMENT) body += `#${earnings.TIP_COMMENT.bestRank} in comment zapping with ${fmt(earnings.TIP_COMMENT.msats)} in total`
|
2023-10-13 19:30:58 +00:00
|
|
|
|
|
|
|
return { title, tag, body }
|
|
|
|
}
|