0.11.2
An initiative (src) DEPENDS ON (verb) a dependency (dst). Forward: depending on something shows the value of the dependency. Backward: having a dependency does not endorse the iniative, but does flow some cred to incentivize reuse and attribution.
Type: EdgeType
An initiative (src) REFERENCES (verb) a reference (dst). Forward: referencing from an initiative shows the value of the reference. But we assume a reference likely needs some refinement to be used by the initiative, so it flows less cred than to a dependency. Backward: having reference material does not endorse the iniative, but does flow some cred to incentivize using existing research and attribution.
Type: EdgeType
A contribution (src) CONTRIBUTES TO (verb) an initiative (dst). Forward: a contribution towards the initiative is also an endorsement of the value of that initiative. Backward: an initiative in large part consists of it's contributions, so the value of an initiative caries over to it's contributions.
Type: EdgeType
A contributor (src) CONTRIBUTES TO (verb) an entry node (dst). Forward: a contributor towards the entry node has a small endorsement of that contribution. Though a high weight would risk contributors' own cred gets "lost to alpha". Backward: flows the value of the contribution to the contributors.
Type: EdgeType
A user (src) CHAMPIONS (verb) an initiative (dst). Meaning forward is the user claiming and committing they will champion an initiative. And backward is the return of cred based on the completion and succesful championing of the ininiative.
Forward: a user championing an iniative is also an endorsement of the value of that initiative. Backward: an initiative likely received a lot of ongoing support from it's champion. We're assuming this is more support than individual contributions.
Type: EdgeType
Return the address corresponding to a GitHub login.
If the login is considered a bot, then a bot address is returned. Otherwise, a regular user address is returned. The method does not attempt to find out whether the address should actually be an organization address, as we don't yet handle organization addresses.
Note: The signature will need to be refactored when we make the list of bots a configuration option rather than a hardcoded constant.
(string)
RawAddress
parseAddress will accept any 20-byte hexadecimal ethereum address encoded as
a string, optionally prefixed with 0x
.
Per EIP-55 (https://eips.ethereum.org/EIPS/eip-55), parseAddress throws if the provided string is mixed-case but not checksum-encoded. All valid addresses in lower- and upper-case format will not throw.
For consistency, all valid addresses are converted and returned in
mixed-case form with the 0x
prefix included
valid formats: "2Ccc7cD913677553766873483ed9eEDdB77A0Bb0" "0x2Ccc7cD913677553766873483ed9eEDdB77A0Bb0" "0X2CCC7CD913677553766873483ED9EEDDB77A0BB0" "0x2ccc7cd913677553766873483ed9eeddb77a0bb0"
invalid formats: "0x2ccc7cD913677553766873483ed9eEDdB77A0Bb0" "2ccc7cD913677553766873483ed9eEDdB77A0Bb0"
(string)
EthAddress
This module defines NodeType
s and EdgeType
s, both of which are
data structures containing shared metadata that describes many nodes or edges
in the graph. Nodes can be "members" of zero or more NodeType
s, and edges can
be "members" of zero or more EdgeType
s. Membership is determined based on the
type's prefix
, which is an address. A node or edge is considered a member of
a type if that type's prefix is a prefix of the node's address.
To make this more concrete, let's consider a specific example. Suppose we define the following nodes:
const pullNode = NodeAddress.fromParts(["github", "pull", "repo", "34"]); const commitNode = NodeAddress.fromParts(["git", "commit", "e1337"]); const pullType: NodeType = { name: "Pull request", prefix: NodeAddress.fromParts(["github", "pull"]), // ... more properties as required }; const githubType: NodeType = { name: "GitHub node", prefix: NodeAddress.fromParts(["github"]) };
Then the pullNode
is considered a member of the pullType
and githubType
,
while the commitNode
is not a member of either type.
The concept of a "type" is useful to SourceCred because it let's us express that groups of nodes are conceptually related, and that we should treat them similarly. Most saliently, we use types to assign default weights to groups of nodes, and to aggregate them for better UI organization. The fact that the SourceCred UI can group all pull requests together, and assign a default weight to all of them, is possible because the GitHub plugin defines a pull request node type.
While a node or edge can theoretically be a member of multiple types, in practice we generally treat the node or edge as though it is only a member of its most specific type. In the example above, we would treat any individual pull request as though it is only a member of the pull request node type. That may change in the future. In general, the type system is not wholly finalized; when it does become finalized, we will likely move it into src/core. See #710 for context.
Represents a "Type" of node in the graph. See the module docstring for context.
Represents a "Type" of edge in the graph. See the module docstring for context.
(string)
(string)
(EdgeWeight)
(EdgeAddressT)
(string)
An IdentityProposal allows a plugin to report a participant identity, for inclusion in the ledger.
The proposal has an alias
, which includes a node address for the identity.
If some account already has that address, then the proposal may be ignored.
If no account has that address, then the proposal will be added as a new identity in the ledger.
The proposal has a proposed name for the identity, and a name for the plugin. The plugin name will be used as a discriminator if there's already a different identity with that name.
If the name and discriminator combo is taken, then a further numeric discriminator will be added.
When the identity is created, it will have its own identity address, per usual, and then the alias will be added. We give the plugin control over the full alias because aliases include helpful descriptions which are shown in the UI, and the plugin should choose an appropriate description.
Given a Ledger and an IdentityProposal, ensure that some Ledger account exists for the proposed identity and return the identity ID.
If there is already an account matching the node address of the proposal's alias, then the ledger is unchanged.
Otherwise, a new account will be created per the semantics of the IdentityProposal type.
(Ledger)
(IdentityProposal)
IdentityId
A dynamic plugin that allows 3rd parties to rapidly pipe data into an instance.
The External plugin can be used multiple times, because it simply uses the PluginId pattern "external/X" where X can be any name (but preferably an agreed upon name between the 3rd-party software and the instance maintainer).
The External plugin loads its graph and optionally its declaration and identityProposals from either:
config/plugins/external/X
folder.
2. a base url that statically serves the files
enabled in the same directory, and add a config.json
file in the
instance's config/plugins/external/X
folder with form:
{ "Url": "https://www.myhost.com/path/to/directory" }
Supported files for either method are:
graph.json
/graph.json.gzip
(required) - works whether or not it iscompressed using our library
2. declaration.json
(optional) - if omitted, a default declaration with
minimal node/edge types is used, but also graphs don't have to adhere to the
declaration if they don't desire to be configured using our
Weight Configuration UI.
3. identityProposals.json
(optional) - if omitted, no identities are proposed
({pluginId: PluginId, storage: DataStorage?, config: ExternalPluginConfig?})
A way for 3rd-party developers to easily test their External Plugin.
After generating a WeightedGraph, a Declaration, and IdentityProposals,
a developer could instantiate a ConstructorPlugin and pass it into our
graph
API using our library in environments such as Observable.
This is a prerequisite for testing using credrank
because of the
IdentityProposals. Once satisfied with the result, they can serve their files
for consumption by an ExternalPlugin configuration.
({weightedGraph: WeightedGraph?, identityProposals: $ReadOnlyArray<IdentityProposal>?, declaration: PluginDeclaration?, pluginId: PluginId?})
The ZipStorage class composes with other WritableDataStorage implementations.
It compresses value
s before passing them into the underlying baseStorage
implementation, and decompresses them upon receit from baseStorage
.
(DataStorage)
This class serves as a simple wrapper for http GET requests using fetch. If an empty string is passed as the base, the base will be interpretted as '.'
(string)
This get method will error if a non-200 or 300-level status was returned, or if the resource traverses above the base path.
(string)
Promise<Uint8Array>
A Name is an identity name which has the following properties:
Parse a Name from a string.
Throws an error if the Name is invalid.
(string)
Name
Attempt to coerce a string into a valid name, by replacing invalid
characters like _
or #
with hyphens.
This can still error, if given a very long string or the empty string, it will fail rather than try to change the name length.
(string)
Name
Assert at runtime that the provided address is actually a valid
address of this kind, throwing an error if it is not. If what
is
provided, it will be included in the error message.
(Address)
(string?)
void
Assert at runtime that the provided array is a valid array of
address parts (i.e., a valid input to fromParts
), throwing an
error if it is not. If what
is provided, it will be included in
the error message.
void
The empty address (the identity for append
). Equivalent to
fromParts([])
.
Type: Address
Convert an array of address parts to an address. The input must be
a non-null array of non-null strings, none of which contains the
NUL character. This is the inverse of toParts
.
($ReadOnlyArray<string>)
Address
Convert an address to the array of parts that it represents. This
is the inverse of fromParts
.
(Address)
Array<string>
Pretty-print an address. The result will be human-readable and contain only printable characters. Clients should not make any assumptions about the format.
(Address)
string
Construct an address by extending the given address with the given additional components. This function is equivalent to:
return fromParts([...toParts(address), ...components]);
but may be more efficient.
Address
Test whether the given address has the given prefix. This function is equivalent to:
const prefixParts = toParts(prefix);
const addressParts = toParts(address);
const actualPrefix = addressParts.slice(0, prefixParts.length);
return deepEqual(prefix, actualPrefix);
(where deepEqual
checks value equality on arrays of strings), but
may be more efficient.
Note that this is an array-wise prefix, not a string-wise-prefix:
e.g., toParts(["ban"])
is not a prefix of toParts(["banana"])
.
(Address)
(Address)
boolean
Interpret the provided string as an Address.
Addresses are natively stored as strings. This method verifies that the provided "raw" address is actually an Address, so that you can have a type-level assurance that a string is an Address.
This is useful if e.g. you are loading serialized Addresses.
Throws an error if the string is not a valid Address.
(string)
Address
A parser for Addresses.
Convenience wrapper fromRaw
.
Type: C.Parser<Address>
The name of this kind of address, like NodeAddress
.
Type: string
A unique nonce for the runtime representation of this address. For compact serialization, this should be short; a single letter suffices.
Type: string
For the purposes of nice error messages: in response to an address of the wrong kind, we can inform the user what kind of address they passed (e.g., "expected NodeAddress, got EdgeAddress"). This dictionary maps another address module's nonce to the name of that module.
Convert a string-keyed map to an object. Useful for conversion to
JSON. If a map's keys are not strings, consider invoking mapKeys
first.
($ReadOnlyMap<InK, InV>)
{}
Convert an object to a map. The resulting map will have key-value
pairs corresponding to the enumerable own properties of the object in
iteration order, as returned by Object.keys
.
({})
Map<K, V>
Shallow-copy a map, allowing upcasting its type parameters.
The Map
type constructor is not covariant in its type parameters,
which means that (e.g.) Map<string, Dog>
is not a subtype of
Map<string, Animal>
even if Dog
is a subtype of Animal
. This is
because, given a Map<string, Animal>
, one can insert a Cat
, which
would break invariants of existing references to the variable as a
map containing only Dog
s.
declare class Animal {};
declare class Dog extends Animal {};
declare class Cat extends Animal {};
declare var dogMap: Map<string, Dog>;
const animalMap: Map<string, Animal> = dogMap; // must fail
animalMap.set("tabby", new Cat()); // or we could do this...
(dogMap.values(): Iterator<Dog>); // ...now contains a `Cat`!
This problem only exists when a map with existing references is
mutated. Therefore, when we shallow-copy a map, we have the
opportunity to upcast its type parameters: copy(dogMap)
can be a
Map<string, Animal>
.
($ReadOnlyMap<InK, InV>)
Map<K, V>
Map across the keys of a map. Note that the key-mapping function is provided both the key and the value for each entry.
The key-mapping function must be injective on the map's key set. If it maps two distinct input keys to the same output key, an error may be thrown.
($ReadOnlyMap<InK, InV>)
(function (InK, InV): K)
Map<K, V>
Map across the values of a map. Note that the value-mapping function is provided both the key and the value for each entry.
There are no restrictions on the value-mapping function (in particular, it need not be injective).
($ReadOnlyMap<InK, InV>)
(function (InK, InV): V)
Map<K, V>
Map simultaneously across the keys and values of a map.
The key-mapping function must be injective on the map's key set. If it maps two distinct input keys to the same output key, an error may be thrown. There are no such restrictions on the value-mapping function.
($ReadOnlyMap<InK, InV>)
(function (InK, InV): [K, V])
Map<K, V>
Merge maps without mutating the arguments.
Merges multiple maps, returning a new map which has every key from the source maps, with their corresponding values. None of the inputs are mutated. In the event that multiple maps have the same key, an error will be thrown.
($ReadOnlyArray<$ReadOnlyMap<K, V>>)
Map<K, V>
Merge multiple WeightedGraphs together.
This delegates to the semantics of Graph.merge and Weights.merge.
($ReadOnlyArray<WeightedGraph>)
WeightedGraph
Merge multiple Weights together.
The resultant Weights will have every weight specified by each of the input weights.
When there are overlaps (i.e. the same address is present in two or more of the Weights), then the appropriate resolver will be invoked to resolve the conflict. The resolver takes two weights and combines them to return a new weight.
When no resolvers are explicitly provided, merge defaults to conservative "error on conflict" resolvers.
($ReadOnlyArray<WeightsT>)
({nodeResolver: NodeOperator, edgeResolver: EdgeOperator}?)
WeightsT
Given a map whose values are arrays, push an element onto the array corresponding to the given key. If the key is not in the map, first insert it with value a new empty array.
If the key is already in the map, its value will be mutated, not replaced.
Array<V>
Given a Map, transform its entries into an Array using a provided transformer function.
($ReadOnlyMap<K, V>)
(function (pair: [K, V], index: number): R)
Array<R>
This module contains the Graph, which is one of the most fundamental pieces of SourceCred. SourceCred uses this graph to model all of the contributions that make up a project, and the relationships between those contributions.
If you aren't familiar with computer science graphs, now would be a good time to refresh. See this StackOverflow answer for an introduction, and Wikipedia for a more thorough overview. This Graph is used by SourceCred as a "Contribution Graph", where every node is a contribution or contributor (e.g. a pull request, or a GitHub user identity) and every edge represents a connection between contributions or contributors (e.g. a pull request contains a comment, or a comment is authored by a user).
The Graph serves a simple function: it keeps track of which Nodes exist, and
what Edges join those nodes to each other. Nodes and Edges are both identified
by Addresses; specifically, NodeAddressT
s and EdgeAddressT
s.
In both cases, addresses are modeled as arrays of strings. For example, you might want to give an address to your favorite node. You can do so as follows:
const myAddress: NodeAddressT = NodeAddress.fromParts(["my", "favorite"])
Edge Addresses are quite similar, except you use the EdgeAddress module.
We model addresses as arrays of strings so that plugins can apply hierarchical namespacing for the address. In general, for any address, the first piece should be the name of the organization that owns the plugin, and the second piece should be the name of the plugin. Pieces thereafter are namespaced by the plugin's internal logic. For example, SourceCred has a Git plugin, and that plugin produces addresses like ["sourcecred", "git", "commit", "9cba0e9e212a287ce26e8d7c2d273e1025c9f9bf"].
This enables "prefix matching" for finding only certain types of nodes. For example, if we wanted to find every Git commit in the graph, we could use the following code:
const commitPrefix = NodeAddress.fromParts(["sourcecred", "git", "commit"]); const commitNodes = graph.nodes({prefix: commitPrefix});
The graph represents nodes as the Node
data type, which includes an
address (NodeAddressT) as well as a few other fields that are needed for
calculating and displaying cred. The Graph is intended to be a lightweight
data structure, so only data directly needed for cred analysis is included.
If there's other data you want to store (e.g. the full text of posts that
are tracked in the graph), you can use the node address as a key for a
separate database.
Edges are represented by Edge
objects. They have src
and dst
fields.
These fields represent the "source" of the edge and the "destination" of the
edge respectively, and both fields contain NodeAddressT
s. The edge also
has its own address, which is an EdgeAddressT
.
Graphs are allowed to contain Edges whose src
or dst
are not present.
Such edges are called 'Dangling Edges'. An edge may convert from dangling to
non-dangling (if it is added before its src or dst), and it may convert from
non-dangling to dangling (if its src or dst are removed).
Supporting dangling edges is important, because it means that we can require metadata be present for a Node (e.g. its creation timestamp), and still allow graph creators that do not know a node's metadata to create references to it. (Of course, they still need to know the node's address).
Here's a toy example of creating a graph:
Graph has a number of accessor methods:
hasNode
to check if a node address is in the Graphnode
to retrieve a node by its addressnodes
to iterate over the nodes in the graphhasEdge
to check if an edge address is in the GraphisDanglingEdge
to check if an edge is danglingedge
to retrieve an edge by its addressedges
to iterate over the edges in the graphneighbors
to find all the edges and nodes adjacent to a node
(also supports filtering by direction, by node prefix, or edge prefix)The Graph also has a few other convenience methods, like toJSON/fromJSON
for serialization, and Graph.merge
for combining multiple graphs.
Represents a node in the graph.
An edge between two nodes.
Specifies how to contract a graph, collapsing several old nodes into a single new node, and re-writing edges for consistency.
($ReadOnlyArray<NodeAddressT>)
(Node)
This module adds a system for specifying "bonus minting" policies. The core idea for bonus minting is that extra Cred is conjured out of thin air (as a "bonus") and distributed to a chosen recipient. This system is intended to be used for minting Cred for project-level dependencies. For example, we would like users of SourceCred to mint some extra Cred and flow it to the SourceCred project.
In CredRank, we handle this by creating extra nodes in the graph which mint the bonus Cred, and it flows directly from those nodes to the intended recipients.
The total amount of Cred that may be minted is unbounded; for example, if the dependencies have a total weight of 0.2, then the total Cred will be 120% of the base Cred, but if the dependencies had a total weight of 1, then the total Cred would be double the base Cred. This was a deliberate design decision so that dependency minting would feel "non-rival", i.e. there is not a fixed budget of dependency cred that must be split between the dependencies. In some cases, it may be reasonable for the total Cred flowing to a project's dependencies to be larger than the total Cred flowing directly to the project's contributors; consider that the total amount of time/effort invested in building all the dependencies may be orders of magnitude larger than investment in the project itself.
Compute the union of the given graphs. The result is a new graph that has all of the nodes and all of the edges from all the provided graphs.
If two of the given graphs have edges with the same address, the edges must be equal (i.e. must have the same source and destination in each graph). If this is not the case, an error will be thrown.
Example usage:
const g1 = new Graph().addNode(a).addNode(b).addEdge(e); const g2 = new Graph().addNode(b).addNode(c).addEdge(f); const g3 = Graph.merge([g1, g2]); Array.from(g3.nodes()).length; // 3 Array.from(g3.edges()).length; // 2 const g1 = new Graph().addNode(a).addNode(b).addEdge(x); const g2 = new Graph().addNode(c); const g3 = Graph.merge([g1, g2]);
The newly created graph is a separate instance from any of the input graphs, and may be mutated independently.
(Iterable<Graph>)
Graph
A node address is 'referenced' if it is either present in the graph, or is the src or dst of some edge.
Referenced nodes always have an entry in this._incidentEdges (regardless of whether they are incident to any edges).
This method ensures that a given node address has a reference.
(NodeAddressT)
A node stops being referenced as soon as it is both not in the graph, and is not incident to any edge. This method must be called after any operation which might cause a node address to no longer be referenced, so that the node can be unreferenced if appropriate.
(NodeAddressT)
Returns how many times the graph has been modified.
This value is exposed so that users of Graph can cache computations over the graph with confidence, knowing that they will be able to check the modification count to know when their cache is potentially invalid.
This value may increase any time the graph is potentially modified, even
if no modification actually occurs; for example, if a client calls
addNode
, the modification count may increase even if the added node was
already present in the graph.
This value is not serialized, and is ignored when checking equality, i.e. two graphs may be semantically equal even when they have different modification counts.
ModificationCount
Remove a node from the graph.
If the node does not exist in the graph, no action is taken and no error is thrown. (This operation is idempotent.)
Removing a node which is incident to some edges is allowed; such edges will become dangling edges. See the discussion of 'Dangling Edges' in the module docstring for details.
Returns this
for chaining.
(NodeAddressT)
this
Test whether there exists a Node corresponding to the given NodeAddress.
This will return false for node addresses which are referenced by some edge, but not actually present in the graph.
(NodeAddressT)
boolean
Returns the Node matching a given NodeAddressT, if such a node exists, or undefined otherwise.
(NodeAddressT)
Node?
Returns an iterator over all of the nodes in the graph.
Optionally, the caller can provide a node prefix. If provided, the iterator will only contain nodes matching that prefix. See semantics of Address.hasPrefix for details.
Clients must not modify the graph during iteration. If they do so, an error may be thrown at the iteration call site.
Nodes are yielded in address-sorted order.
({prefix: NodeAddressT}?)
Iterator<Node>
Add an edge to the graph.
It is permitted to add an edge if its src or dst are not in the graph. See the discussion of 'Dangling Edges' in the module docstring for semantics.
It is an error to add an edge if a distinct edge with the same address already exists in the graph (i.e., if the source or destination are different).
Adding an edge that already exists to the graph is a no-op. (This operation is idempotent.)
Returns this
for chaining.
(Edge)
this
Remove an edge from the graph.
Calling removeEdge on an address that does not correspond to any edge in the graph is a no-op. (This method is idempotent.)
Returns this
for chaining.
(EdgeAddressT)
this
Test whether there is a dangling edge at the given address.
Returns true if the edge is present, and is dangling. Returns false if the edge is present, and is not dangling. Returns undefined if the edge is not present.
See the module docstring for more details on dangling edges.
(EdgeAddressT)
(boolean | any)
Returns an iterator over edges in the graph, optionally filtered by edge address prefix, source address prefix, and/or destination address prefix.
The caller must pass an options object with a boolean field showDangling
,
which determines whether dangling edges will be included in the results.
The caller may also pass fields addressPrefix
, srcPrefix
, and dstPrefix
to perform prefix-based address filtering of edges that are returned.
(See the module docstring for more context on dangling edges.)
Suppose that you want to find every edge that represents authorship by a
user. If all authorship edges have the AUTHORS_EDGE_PREFIX
prefix, and
all user nodes have the USER_NODE_PREFIX
prefix, then you could call:
graph.edges({ showDangling: true, // or false, irrelevant for this example addressPrefix: AUTHORS_EDGE_PREFIX, srcPrefix: USER_NODE_PREFIX, });
In this example, as dstPrefix
was left unset, it will default to
NodeAddress.empty
, which is a prefix of every node address.
Clients must not modify the graph during iteration. If they do so, an error may be thrown at the iteration call site.
The edges are yielded in sorted address order.
(EdgesOptions)
Iterator<Edge>
Find the Neighbors
that are incident to a chosen root node.
A Neighbor
contains an edge that is incident to the root,
and the node at the other endpoint of the edge. This may be
either the source or destination of the edge, depending on whether the
edge is an in-edge or an out-edge from the perspective of the root. For
convenience, a Neighbor
is thus an object that includes both the edge
and the adjacent node.
Every non-dangling edge incident to the root corresponds to exactly one
neighbor, but note that multiple neighbors may have the same node
in the
case that there are multiple edges with the same source and destination.
Callers to neighbors
must provide NeighborsOptions
as follows:
direction: one of Direction.IN, direction.OUT, or Direction.ANY.
nodePrefix: A NodeAddressT to use as a prefix filter for the adjacent node.
If you want all nodes, use NodeAddress
.empty.
edgePrefix: An EdgeAddressT to use as a prefix filter for the edge.
If you want all edges, use EdgeAddress
.empty.
Calling neighbors
on a node that is not present in the graph is an error.
If the root node has an edge for which it is both the source and the
destination (a loop edge), there will be one Neighbor
with the root node
and the loop edge.
No Neighbors
will be created for dangling edges, as such edges do not
correspond to any Node in the graph.
Clients must not modify the graph during iteration. If they do so, an error may be thrown at the iteration call site. The iteration order is undefined.
(NodeAddressT)
(NeighborsOptions)
Iterator<Neighbor>
Serialize a Graph into a plain JavaScript object.
GraphJSON
Create a new graph, in which some nodes have been contracted together.
contractNodes takes a list of NodeContractions, each of which specifies a replacement node, and a list of old node addresses to map onto the new node. A new graph will be returned where the new node is added, none of the old nodes are present, and every edge incident to one of the old nodes has been re-written so that it is incident to the new node instead.
If the same node addresses is "old" for several contractions, all incident edges will be re-written to connect to whichever contraction came last.
If the replacement node is present in the graph, no error will be thrown, provided that the replacement node is consistent with the one in the graph.
If there is a "chain" of remaps (i.e. a->b and b->c), then an error will be thrown, as support for chaining has not yet been implemented.
The original Graph is not mutated.
contractNodes runs in O(n+e+k), where n
is the number of nodes, e
is the
number of edges, and k
is the number of contractions. If needed, we can
improve the peformance by mutating the original graph instead of creating
a new one.
($ReadOnlyArray<NodeContraction>)
Graph
Convert a node into a human readable string.
The precise behavior is an implementation detail and subject to change.
(Node)
string
Convert an edge into a human readable string.
The precise behavior is an implementation detail and subject to change.
(Edge)
string
Convert an edge to an object whose fields are human-readable. This is useful for storing edges in human-readable formats that should not include NUL characters, such as Jest snapshots.
(Edge)
{address: string, src: string, dst: string, timestampMs: TimestampMs}
Load an object from compatibilized state created by toCompat
.
The object has an expected type and version, and may optionally have
handler functions for transforming previous versions into a canonical state.
If a handler is present for the current version, it will be applied.
Throws an error if the compatibilized object is the wrong type, or if its version
is not current and there was no handler for its version.
(CompatInfo)
(Compatible<any>)
({}?)
T
Utilities for working with nullable types: ?T = T | null | void
.
These functions use the native runtime representation, as opposed to
creating an Optional<T>
wrapper class. This ensures that they have
minimal runtime cost (just a function call), and that they are
trivially interoperable with other code.
When a value of type ?T
is null
or undefined
, we say that it is
absent. Otherwise, it is present.
Some functions that typically appear in such libraries are not needed:
join
(??T => ?T
) can be implemented as the identity function,
because the Flow types ??T
and ?T
are equivalent;flatMap
(?T => (T => ?U) => ?U
) can be implemented simply as
map
, again because ??T
and ?T
are equivalent;first
(?T => ?T => ?T
) can be implemented simply as orElse
,
again because ??T
and ?T
are equivalent;isPresent
(?T => boolean
) doesn't provide much value over the
equivalent abstract disequality check;empty
(() => ?T
) and of
(T => ?T
) are
entirely spurious.Other functions could reasonably be implemented, but have been left out because they have rarely been needed:
filter
(?T => (T => boolean) => ?T
);forEach
(?T => (T => void) => void
);orElseGet
(?T => (() => T) => T
), which is useful in the case
where constructing the default value is expensive.(Of these three, orElseGet
would probably be the most useful for
our existing codebase.)
(T?)
(function (T): U)
U?
Apply the given function inside the nullable. If the input is absent, then it will be returned unchanged. Otherwise, the given function will be applied.
(T?)
(function (T): U)
U?
Extract the value from a nullable. If the input is present, it will be returned. Otherwise, an error will be thrown with the provided message (defaulting to the string representation of the absent input).
(T?)
(string?)
T
Extract the value from a nullable. If the input is present, it will be returned. Otherwise, an error will be thrown, with message given by the provided function.
(T?)
(function (): string)
T
Extract the value from a nullable, using the provided default value in case the input is absent.
(T?)
(T)
T
Filter nulls and undefined out of an array, returning a new array.
The functionality is easy to implement without a util method (just call
filter
); however Flow doesn't infer the type of the output array based on
the callback that was passed to filter. This method basically wraps filter
in a type-aware way.
($ReadOnlyArray<T?>)
Array<T>
The WeightedGraph a Graph alongside associated Weights
Any combination of Weights and Graph can make a valid WeightedGraph. If the Weights contains weights for node or edge addresses that are not present in the graph, then those weights will be ignored. If the graph contains nodes or edges which do not correspond to any weights, then default weights will be inferred.
Create a new, empty WeightedGraph.
WeightedGraph
Creates new, empty weights.
WeightsT
Create a new WeightedGraph where default weights have been overriden.
This takes a base WeightedGraph along with a set of "override" weights. The new graph has the union of both the base and override weights; wherever there is a conflict, the override weights will replace the base weights. This is useful in situations where we want to let the user manually specify some weights, and ensure that the user's decisions will trump any defaults.
This method does not mutuate any of the original arguments. For performance reasons, it is not a full copy; the input and output WeightedGraphs have the exact same underlying Graph, which should not be modified.
(WeightedGraph)
(WeightsT)
WeightedGraph
Represents the weight for a particular Node (or node address prefix). Weight 1 is the default value and signifies normal importance. Weights are linear, so 2 is twice as important as 1.
Type: number
Represents the forwards and backwards weights for a particular Edge (or edge address prefix). Weight 1 is the default value and signifies normal importance. Weights are linear, so 2 is twice as important as 1.
Represents the weights for nodes and edges.
The weights are stored by address prefix, i.e. multiple weights may apply to a given node or edge.
(Map<NodeAddressT, NodeWeight>)
(Map<EdgeAddressT, EdgeWeight>)
Return an equivalent form of the given chain whose nodeOrder
is the
provided array, which must be a permutation of the node order of the
original chain.
(OrderedSparseMarkovChain)
($ReadOnlyArray<NodeAddressT>)
OrderedSparseMarkovChain
Return an equivalent form of the given chain such that for for each
node, the entries in chain[node].neighbors
are sorted.
(OrderedSparseMarkovChain)
OrderedSparseMarkovChain
The data inputs to running PageRank.
We keep these separate from the PagerankOptions below, because we expect that within a given context, every call to findStationaryDistribution (or other Pagerank functions) will have different PagerankParams, but often have the same PagerankOptions.
PagerankOptions allows the user to tweak PageRank's behavior, especially around convergence.
A representation of a sparse transition matrix that is convenient for computations on Markov chains.
A Markov chain has nodes indexed from 0
to n - 1
, where n
is
the length of the chain. The elements of the chain represent the
incoming edges to each node. Specifically, for each node v
, the
in-degree of v
equals the length of both chain[v].neighbor
and
chain[v].weight
. For each i
from 0
to the degree of v
(exclusive), there is an edge to v
from chain[v].neighbor[i]
with
weight chain[v].weight[i]
.
In other words, chain[v]
is a sparse-vector representation of
column v
of the transition matrix of the Markov chain.
Type: $ReadOnlyArray<{neighbor: Uint32Array, weight: Float64Array}>
A distribution over the integers 0
through n - 1
, where n
is
the length of the array. The value at index i
is the probability of
i
in the distribution. The values should sum to 1.
Type: Float64Array
Compute the maximum difference (in absolute value) between components in two distributions.
Equivalent to $\norm{pi0 - pi1}_\infty$.
(Distribution)
(Distribution)
number
Data structure representing a particular kind of Markov process, as
kind of a middle ground between the semantic SourceCred graph (in the
core/graph
module) and a literal transition matrix. Unlike the core
graph, edges in a Markov process graph are unidirectional, edge
weights are raw transition probabilities (which must sum to 1) rather
than unnormalized weights, and there are no dangling edges. Unlike a
fully general transition matrix, parallel edges are still reified,
not collapsed; nodes have weights, representing sources of flow; and
a few SourceCred-specific concepts are made first-class:
specifically, cred minting and time period fibration. The
"teleportation vector" from PageRank is also made explicit via the
"adjoined seed node" graph transformation strategy, so this data
structure can form well-defined Markov processes even from graphs
with nodes with no out-weight. Because the graph reifies the
teleportation and temporal fibration, the associated parameters are
"baked in" to weights of the Markov process graph.
We use the term "fibration" to refer to a graph transformation where each scoring node is split into one node per epoch, and incident edges are rewritten to point to the appropriate epoch nodes. The term is vaguely inspired from the notion of a fiber bundle, though the analogy is not precise.
The Markov process graphs in this module have three kinds of nodes:
The edges include:
A Markov process graph can be converted to a pure Markov chain for
spectral analysis via the toMarkovChain
method.
Find an epoch node, or just the original node if it's not a scoring address.
(NodeAddressT)
(TimestampMs)
NodeAddressT
Return the node address's canonical index in the node order, if it is present.
(NodeAddressT)
(number | null)
Returns a canonical ordering of the nodes in the graph.
No assumptions should be made about the node order, other than that it is stable for any given MarkovProcessGraph.
void
Iterate over the nodes in the graph. If a prefix is provided, only nodes matching that prefix will be returned.
The nodes are always iterated over in the node order.
({prefix: NodeAddressT}?)
void
Return the edge address's canonical index in the edge order, if it is present.
(MarkovEdgeAddressT)
(number | null)
Returns a canonical ordering of the edges in the graph.
No assumptions should be made about the edge order, other than that it is stable for any given MarkovProcessGraph.
void
Iterate over the edges in the graph.
The edges are always iterated over in the edge order.
void
Yield the canonical node order. This has been separated from the class because we need it at construction time, etc.
($ReadOnlyMap<NodeAddressT, MarkovNode>)
($ReadOnlyArray<TimestampMs>)
($ReadOnlyArray<Participant>)
void
Return an array containing the node addresses for every virtualized node. The order must be stable.
($ReadOnlyArray<TimestampMs>)
($ReadOnlyArray<Participant>)
void
This module allows participants to attribute their cred to other participants. This feature should not be used to make cred sellable/transferable, but instead is intended to allow participants to acknowledge that a portion of their creditted outputs are directly generated/supported by the labor of others. (e.g. when a contributor has a personal assistant working behind the scenes)
(TimestampMs)
(number)
A timestamped configuration representing a decimal proportion of cred flow, which can be applied to a participant pair.
(TimestampMs)
(number)
A recipient of cred attribution and a chronological log of proportion configurations.
(IdentityId)
($ReadOnlyArray<PersonalAttributionProportion>)
A participant that is attributing their cred, and a log of how they are attributing it.
(IdentityId)
($ReadOnlyArray<AttributionRecipient>)
A list of participants who are attributing their cred, with logs of how they are attributing it.
Type: $ReadOnlyArray<PersonalAttribution>
Validates that:
(PersonalAttributions)
This is the intermediary data structure used to index personal attributions data, making lookups faster. It can be interpreted as:
$ReadOnlyMap< fromParticipantId, $ReadOnlyMap<toParticipantId, AttributionRecipient>
Type: $ReadOnlyMap<IdentityId, $ReadOnlyMap<IdentityId, AttributionRecipient>>
An indexed store of PersonalAttributions that includes optimized queries needed by credrank.
(PersonalAttributions)
($ReadOnlyArray<TimestampMs>)
Validates that:
(Index)
($ReadOnlyArray<TimestampMs>)
Returns a non-indexed, json-friendly PersonalAttributions. The order may be changed from the original PersonalAttributions that was used to construct this object, but the elements are the same and the order is generated consistently.
PersonalAttributions
Return the IDs of all of the recipients that receive a non-zero proportion of the given participant's cred in the given epoch.
(TimestampMs)
(IdentityId)
$ReadOnlyArray<IdentityId>
Returns the decimal proportion of the fromParticipant's cred that should flow to the toParticipant in the given epoch.
(number | null)
Returns the total decimal proportion of the fromParticipant's cred that should flow to other participants in the given epoch.
(TimestampMs)
(IdentityId)
(number | null)
An analytics-only timestamp. Not built for continued functionality within a MarkovProcessGraph (where epoch nodes are generated and used instead).
Type: (TimestampMs | null)
This module contains logic for creating nodes and edges that act as "gadgets" in CredRank. They are most directly used by markovProcessGraph.js
A helper function for creating a gadget only produces edges incident to seed. We assume that it has a function for converting from the target type into node address parts, which will be used to produce a unique edge address, and which are the address parts for the src or dst. If seedIsSrc is true, then the seed is the src and the dst will be the target. Otherwise, the seed is the dst and the target will be the src. These markov edges are never reversed.
(MakeSeedGadgetArgs<T>)
Name | Description |
---|---|
$0.edgePrefix any
|
|
$0.seedIsSrc any
|
|
$0.toParts any
|
|
$0.fromParts any
|
EdgeGadget<T>
The payout gadget creates edges that connect participant epoch nodes to the epoch accumulator nodes. Each payout edge represents the flow of Cred from a participant's epoch back to the seed (by means of the accumulator). Thus, the Cred flow on this edge actually represents Cred score for the participant. (The Cred score of the epoch node can't be seen as the user's score, because some of it flows to other contributions, to other epoch nodes, etc.)
Type: EdgeGadget<ParticipantEpochAddress>
The forward webbing edges flow Cred forwards from participant epoch nodes to the temporally next epoch node from the same participant. The intention is to "smooth out" Cred over time by having some of it flow forwards in time.
Type: EdgeGadget<WebbingAddress>
The backward webbing edges flow Cred backwards from participant epoch nodes to the temporally previous epoch node from the same participant. The intention is to "smooth out" Cred over time by having some of it flow backwards in time.
Type: EdgeGadget<WebbingAddress>
This module exposes a class that accesses participant data, aggregating between a CredGraph and a Ledger.
It is useful for cases where you want to view a participant's Cred and Grain data simultaneously, for example for creating summary dashboards.
This module outputs aggregated data that combines Cred Scores with Ledger Account data.
We use this internally when creating Grain distributions using a Ledger and a Cred View. It's also an experimental output format which gives overall information on the cred in an instance. We may remove it or make breaking changes to it in the future.
Sum a sequence of Grain values.
(Iterable<Grain>)
Grain
Cred and Grain data for a given participant.
Implicitly has an associated time scope, which will be the time scope of the CredGrainView or TimeScopedCredGrainView that generated this.
The indices of credPerInterval/grainEarnedPerInterval correspond to the same indices in the IntervalSequence of the CredGrainView or TimeScopedCredGrainView that generated this.
Aggregates data across a CredGraph and Ledger.
By default, it includes data across all time present in the instance.
Callers can call withTimeScope
to get a TimeScopedCredGrainView
which
returns data that only includes a continuous subset of cred and grain data
across time.
($ReadOnlyArray<ParticipantCredGrain>
= []
)
(IntervalSequence
= intervalSequence([])
)
Creates a CredGrainView using the output of the CredRank API.
(CredGraph)
(Ledger)
CredGrainView
Creates a CredGrainView using the output of the CredEquate API.
CredGrainView
Combines multiple CredGrainViews into a single one. The intervals will span the earliest to the latest of all intervals in all the CredGrainViews. Participant identity/active data will match that of the first occurrence of that participant in the params.
(...$ReadOnlyArray<CredGrainView>)
CredGrainView
Returns the data for the participant with the given ID
(IdentityId)
ParticipantCredGrain
This class's contructor stores a continuous subset of the originalIntervals and participant cred/grain data where intervals are only included if their start and end times are both within the provided startTimeMs and endTimeMs, inclusively.
Represents a time interval The interval is half open [startTimeMs, endTimeMs), i.e. if a timestamp is exactly on the interval boundary, it will fall at the start of the older interval.
(TimestampMs)
(TimestampMs)
An interval sequence is an array of intervals with the following guarantees:
Represents a slice of a time-partitioned graph Includes the interval, as well as all of the nodes and edges whose timestamps are within the interval.
Partition a graph based on time intervals.
The intervals are always one week long, as calculated using d3.utcWeek. The result may contain empty intervals. If the graph is empty, no intervals are returned. Timeless nodes are not included in the partition, nor are dangling edges.
(Graph)
GraphIntervalPartition
Produce an array of Intervals which cover all the node and edge timestamps for a graph.
The intervals are one week long, and are aligned on clean week boundaries.
This function is basically a wrapper around weekIntervals that makes sure the graph's nodes and edges are all accounted for properly.
(Graph)
IntervalSequence
Produce an array of week-long intervals to cover the startTime and endTime.
Each interval is one week long and aligned on week boundaries, as produced by d3.utcWeek. The weeks always use UTC boundaries to ensure consistent output regardless of which timezone the user is in.
Assuming that the inputs are valid, there will always be at least one interval, so that that interval can cover the input timestamps. (E.g. if startMs and endMs are the same value, then the produced interval will be the start and end of the last week that starts on or before startMs.)
IntervalSequence
Sorting utility. Accepts an array and optionally any number of "pluck" functions to get the value to sort by. Will create a shallow copy, and sort in ascending order.
arr
: The input array to sortpluckArgs
: (0...n) Functions to get the value to sort by. Defaults to identity.($ReadOnlyArray<T>)
(...Array<PluckFn<T>>)
Array<T>
This module contains the ledger, for accumulating state updates related to identity identities and Grain distribution.
A key requirement for the ledger is that we need to store an ordered log of every action that's happened in the ledger, so that we can audit the ledger state to ensure its integrity.
Timestamped record of a grain payment made to an Identity from a specific Allocation.
The state of the Ledger's accounting configuration.
Every Identity in the ledger has an Account.
(Identity)
(G.Grain)
(G.Grain)
(Array<AllocationReceipt>)
(boolean)
(PayableAddressStore)
(Array<IdentityId>)
The current Grain balance of this account. All balances are zero when accounting is disabled.
Type: G.Grain
PayableAddressStore maps currencies to a participant's address capable of accepting the currency. This structure exists to accomodate safe migration for grain/payout token changes. Users must verify themselves that the address they are supplying is capable of receiving their share of a grain distribution.
Type: Map<CurrencyKey, PayoutAddress>
The Ledger is an append-only auditable data store which tracks
Every time the ledger state is changed, a corresponding Action is added to
the ledger's action log. The ledger state may be serialized by saving the
action log, and then reconstructed by replaying the action log. The
corresponding methods are actionLog
and Ledger.fromActionLog
.
None of these methods are idempotent, since they all modify the Ledger state on success by adding a new action to the log. Therefore, they will all fail if they would not cause any change to the ledger's logical state, so as to prevent the ledger from permanently accumulating no-op clutter in the log.
It's important that any API method that fails (e.g. trying to add a conflicting identity) fails without mutating the ledger state; this way we avoid ever getting the ledger in a corrupted state. To make this easier to test, the test code uses deep equality testing on the ledger before/after attempting illegal actions. To ensure that this testing works, we should avoid adding any ledger state that can't be verified by deep equality checking (e.g. don't store state in functions or closures that aren't attached to the Ledger object).
Every Ledger action has a timestamp, and the Ledger's actions must always be in timestamp-sorted order. Adding a new Action with a timestamp older than a previous action is illegal.
Return all the Accounts in the ledger.
$ReadOnlyArray<Account>
Get the Account associated with a particular identity.
If the identity is not in the ledger, an error is thrown.
(IdentityId)
Account
Return the account matching a given NodeAddress, if one exists.
Returns null if there is no account matching that address.
(NodeAddressT)
(Account | null)
Get the Allocation associated with a particular Allocation ID.
If the ID is not in the ledger, an error is thrown.
(AllocationId)
Allocation
Get an Iterator over all Allocations in the order they occur in the Ledger.
Iterator<Allocation>
Get the Distribution associated with a particular Distribution ID.
If the ID is not in the ledger, an error is thrown.
(DistributionId)
Distribution
Get an Iterator over all Distributions in the order they occur in the Ledger.
Iterator<Distribution>
Get the Distribution associated with a particular Allocation ID.
If the Allocation ID is not associated with a distribution, an error is thrown.
(AllocationId)
Distribution
Create an account in the ledger.
This will reserve the identity's name, and its innate address.
This returns the newly created Identity's ID, so that the caller store it for future reference.
Will fail if the name is not valid, or already taken.
(IdentityType)
(string)
IdentityId
Merge two identities together.
One identity is considered the "base" and the other is the "target". The target is absorbed into the base, meaning:
Attempting to merge an identity that doesn't exist, or to merge an identity into itself, will error.
({base: IdentityId, target: IdentityId})
Ledger
Add an alias for a identity.
If that alias is associated with past Grain payments (because it was unlinked from another identity), those past Grain payments will be associated with the newly linked identity.
Will fail if the identity does not exist. Will fail if the alias is already claimed by any identity.
(IdentityId)
(Alias)
Ledger
Deactivate an account, making it ineligible to send or recieve Grain.
The account's Grain balance will remain untouched until it is reactivated.
If the account is already inactive, this will no-op (without emitting any event).
(IdentityId)
Ledger
Canonicalize a Grain distribution in the ledger.
Fails if any of the recipients are not active.
(Distribution)
Ledger
Transfer Grain from one account to another.
Fails if the sender does not have enough Grain, or if the Grain amount is negative. Fails if either the sender or the receipient have not been activated. Self-transfers are supported. An optional memo may be added.
Note: The arguments need to be bundled together in an object with named
keys, to avoid getting confused about which positional argument is from
and which one is to
.
({from: IdentityId, to: IdentityId, amount: G.Grain, memo: (string | null)})
this
setPayoutAddress allows participants to set a payable address to collect grain. These addresses are keyed on a specific currency, which ensures that users don't erroneously receive a grain distribution to an address that cannot handle it (such as a custodial wallet, or rigidly-designed smart contract) and effectively lose that reward.
An address may be deleted by passing in null
for the
payoutAddress
parameter. This is useful in case the underlying private key
is compromised or the exchange hosting a custodial account is hacked.
(IdentityId)
((PayoutAddress | null))
(ChainId)
(EthAddress?)
Ledger
Enabling Integration Tracking has the following effects:
markDistributionAsExecuted
is called on it.trackedDistributions
getter will return an array of distributionIds
created since tracking was enabled. Their statuses can be queried using
isGrainIntegrationExecuted
.Integration Tracking can be enabled and disabled at any time. Some integrations might error if tracking is not enabled, to ensure their status is tracked in the ledger. This allows for the decoupling of (1) calculating a grain distribution and then (2) actually distributing it via the grain integration.
No-ops if integration tracking is already enabled
Ledger
Setting a currency is a prerequisite to executing a grain integration.
When accounting is enabled, this function will set an external currency in the ledger, but will not modify any accounts.
When accounting is disabled, this function can be called to update the ledger's external currency configuration. Note that this will also deactivate identities missing a payout address that matches the new configuration when accounting is disabled.
(ChainId)
(EthAddress?)
Ledger
Returns the state of the ledger's accounting configuration.
AccountingStatus
Expectably returns the currency from the externalCurrency attribute. It throws if a currency is not set.
Currency
When integration tracking is disabled, an empty iterator is returned.
When integration tracking is enabled, the array of tracked
DistributionIds
is returned
The iterator is cleared each time
disableIntegrationTracking
is called.
Iterator<DistributionId>
Returns the status of a tracked distribution
If the distribution has not been executed by the integration, false
is
returned.
If the distribution has been executed, true
is returned.
If the distribution is untracked, undefined
is returned.
(DistributionId)
boolean?
Retrieve the log of all actions in the Ledger's history.
May be used to reconstruct the Ledger after serialization.
LedgerLog
Return the cred-effective timestamp for the last Grain distribution.
We provide this because we may want to have a policy that issues one distribution for each interval in the history of the project.
If there were never any distributions, then null will be returned.
(TimestampMs | null)
Helper method for deactivating accounts that don't have a payout address
set for the configured currencyKey
void
Helper method for disabling accounting.
Zero out all balances, so there is no confusion about them. By zeroing them out, they cannot be compared to external balances in any meaningful way.
void
The Actions are used to store the history of Ledger changes.
Type: (CreateIdentity | RenameIdentity | AddAlias | MergeIdentities | ToggleActivation | DistributeGrain | TransferGrain | ChangeIdentityType | SetPayoutAddress | EnableGrainIntegration | DisableGrainIntegration | MarkDistributionExecuted | EnableAccounting | DisableAccounting | SetExternalCurrency | RemoveExternalCurrency)
EvmChainId is represented in the form of a stringified integer for all EVM-based chains, including mainnet (1), and xDai (100). The reason for this is that ethereum's client configuration utilizes a number to represent chainId, and this way we can just transpose that chainId here as a component of the currency Id, since the web3 client will return a stringified integer when the chainId is requested.
tokenAddress is a subset of all available EthAddresses.
A token address is the address of the token contract for an ERC20 token, or the 20 byte-length equivalent of 0x0, which is the conventional address used to represent ETH on the ethereum mainnet, or the native currency on an EVM-based sidechain. See here for more details on these semantics: https://ethereum.org/en/developers/docs/intro-to-ethereum/#eth
Type: EthAddress
Example protocol symbols: "BTC" for bitcoin and "FIL" for Filecoin
Chains like Bitcoin and Filecoin do not have "production" sidechains so we represent them as a string, as specified in the ProtocolSymbol type
("PROTOCOL"
)
(ProtocolSymbol)
The Currency key must be stringified to ensure the data is retrievable. Keying on the raw Currency object means keying on the object reference, rather than the contents of the object.
Generate a uniformly random clean ID.
Uuid
Parse a serialized UUID. This is the left inverse of the trivial
injection from Uuid
to string
, and throws on invalid input.
(string)
Uuid
Parse a serialized UUID. This expects to parse a JSON string value
with the same semantics as fromString
.
Type: C.Parser<Uuid>
Fill the given buffer with cryptographically secure random bytes. The buffer length must not exceed 65536.
(Uint8Array)
Uint8Array
In SourceCred, projects regularly distribute Grain to contributors based on their Cred scores. This is called a "Distribution". This module contains the logic for computing distributions.
JsonLog tracks and serializes append-only logs of JSON values.
At its heart, it's basically a simple wrapper around an array, which enforces the rule that items may be appended to it, but never removed.
It also provides serialization logic. We store the log as a
newline-delimited stream of JSON values, with a one-to-one correspondence
between POSIX lines and elements in the sequence. That is, the serialized
form of an element will never contain an embedded newline, and there are no
empty lines. JSON streams can be easily inspected and manipulatedwith tools
like jq
as well as standard Unix filters, and can be stored and
transmitted efficiently in Git repositories thanks to packfiles and delta
compression.
Elements of a JsonLog
are always parsed using a Combo.Parser, which
ensures type safety at runtime.
Iteratively compute and distribute Grain, based on the provided CredGraph, into the provided Ledger, according to the specified DistributionPolicy.
Here are some examples of how it works:
The last time there was a distribution was two days ago. Since then, no new Cred Intervals have been completed. This method will no-op.
The last time there was a distribution was last week. Since then, one new Cred Interval has been completed. The method will apply one Distribution.
The last time there was a distribution was a month ago. Since then, four Cred Intervals have been completed. The method will apply four Distributions, unless maxOldDistributions is set to a lower number (e.g. 2), in which case that maximum number of distributions will be applied.
It returns the list of applied distributions, which may be helpful for diagnostics, printing a summary, etc.
(GrainConfig)
(CredGrainView)
(Ledger)
(TimestampMs)
(boolean)
$ReadOnlyArray<Distribution>
Compute a single Distribution using CredAccountData.
The distribution will include the provided policies. It will be computed using only Cred intervals that are finished as of the effectiveTimestamp.
Note: This method is untested as it is just a bit of plubming; flow gives me confidence that the semantics are correct. *
($ReadOnlyArray<AllocationPolicy>)
(CredGrainView)
(TimestampMs)
Distribution
This module contains the types for tracking Grain, which is the native project-specific, cred-linked token created in SourceCred instances. In practice, projects can call these tokens anything they want, but we will refer to the tokens as "Grain" throughout the codebase. The conserved properties of all Grains are that they are minted/distributed based on cred scores, and that they can be used to Boost contributions in a cred graph.
We track Grain using big integer arithmetic, so that we can be precise with Grain values and avoid float imprecision issues. Following the convention of ERC20 tokens, we track Grain at 18 decimals of precision, although we can make this project-specific if there's a future need.
At rest, we represent Grain as strings. This is a convenient decision around serialization boundaries, so that we can just directly stringify objects containing Grain values and it will Just Work. The downside is that we need to convert them to/fro string representations any time we need to do Grain arithmetic, which could create perf hot spots. If so, we can factor out the hot loop and do them in a way that has less overhead. You can see context for this decision in #1936 and #1938.
Ideally, we would just use the native BigInt type. However, at time of writing it's not well supported by flow or Safari, so we use the big-integer library. That library delegates out to native BigInt when available, so this should be fine.
Since the big-integer library does have a sensible toString
method defined
on the integers, we could switch to representing Grain at rest via
big-integers rather than as strings. However, this would require re-writing
a lot of test code. If perf becomes an issue that would be a principled fix.
Formats a grain balance as a human-readable number, dividing the
raw grain balance by one
.
The client controls how many digits of precision are shown; by default, we display zero digits. Grain balances will have commas added as thousands-separators if the balance is greater than 1000g.
The client also specifies a suffix; by default, we use 'g' for grain.
Here are some examples of its behavior, pretending that we use 2 decimals of precision for readability:
format(133700042n) === "1,337,000g" format(133700042n, 2) === "1,337,000.42g" format(133700042n, 2, "seeds") === "1,337,000.42seeds" format(133700042n, 2, "") === "1,337,000.42"
string
Formats a grain balance as a human-readable number using the format() method, but trims any unnecessary decimal information.
The intended use is for UI presentation where less visual clutter is desired.
Here are some examples of its behavior
formatAndTrim(100000000000000) === "0.0001g" formatAndTrim(150000000000000000000) === "150g" formatAndTrim(15000000000000000000000) === "15,000g" formatAndTrim(15000000000000000000000, "seeds") === "15,000seeds" formatAndTrim(15000000000000000000000, "") === "15,000"
string
Multiply a grain amount by a floating point number.
Use this method when you need to multiply a grain balance by a floating point number, e.g. a ratio.
Note that this method is imprecise. It is not safe to assume, for example,
that multiply(g, 1/3) + multiply(g, 2/3) === g
due to loss of precision.
However, the errors will be small in absolute terms (i.e. tiny compared to
one full grain).
See some messy analysis of the numerical errors here: https://observablehq.com/@decentralion/grain-arithmetic
Grain
Convert an integer number (in floating-point representation) into a precise Grain value.
(number)
Grain
Accept human-readable numbers strings and convert them to precise grain amounts
This is most useful for processing form input values before passing them into the ledger, since all form fields return strings
In this case, a "float string" is a string that returns a number value
when passed into parseFloat
The reason to circumvent any floating point values is to avoid losses in precision. By modifying the string directly in a predictable pattern, we can convert uer-generated floating point values to grain at full fidelity, and avoid any fuzzy floating point arithmetic
The tradeoff here is around versatility. Values with more decimals than the allowable precision will yield an error when passed in.
Grain
Approximately create a grain balance from a float.
This method tries to convert the floating point amt
into a grain
balance. For example, grain(1)
approximately equals ONE
.
Do not assume this will be precise! For example, grain(0.1337)
results in
133700000000000016n
. This method is intended for test code.
This is a shorthand for multiplyFloat(ONE, amt)
.
(number)
Grain
Approximates the division of two grain values
This naive implementation of grain division converts the given values to floats and performs simple floating point division.
Do not assume this will be precise!
number
Splits a budget of Grain proportional to floating-point scores.
splitBudget guarantees that the total amount distributed will precisely equal the budget. This is a surprisingly challenging property to ensure, and it explains the complexity of this algorithm. We stress-test the method with extremely uneven share distribution (e.g. a split where some users' scores are 10**100 larger than others).
The algorithm can be arbitrarily unfair at the atto-Grain level; for
example, in the case splitBudget(fromString("1"), [1, 100])
it will give
all the Grain to the first account, even though it only has 1/100th the score
of the second account. However, since Grain is tracked with 18 decimal point
precision, these tiny biases mean very little in practice. In testing, when
splitting one full Grain (i.e. 10**18 attoGrain), we haven't seen discrepancies
over ~100 attoGrain, or one billion-million-th of a full Grain.
$ReadOnlyArray<Grain>
Uncomment below if you want to measure the discrepancy caused by forcing fracion=1 whenever fraction > 1.
In testing, when distributing one full Grain across wildly unequal scores, it never produced more than ~hundreds of attoGrain discrepancy.
Uncomment below if you want to measure the discrepancy caused by this "giveaway-leftovers" approach. In testing, when run with wildly varying shares, it never produced more than ~hundreds of attoGrain discrepancy.
Shape of currencyDetails.json on disk
Shape of currencyDetails json in memory after parsing
Utilized by combo.fmap to enforce default currency values when parsing. This engenders a "canonical default" since there will be no need to add default fallbacks when handling currency detail values after parsing the serialized file.
CurrencyDetails
An Alias is basically another graph Node which resolves to this identity. We ignore the timestamp because it's generally not significant for users; we keep the address out of obvious necessity, and we keep the description so we can describe this alias in UIs (e.g. the ledger admin panel).
(string)
(NodeAddressT)
A leaf node in the Expression tree structure. It represents a trait that can be weighted atomically.
A recursive type that forms a tree-like structure of algebraic expressions. Can be evaluated as OPERATOR(...weightOperands, ...expressionOperands).
For example, if the operator is ADD, an expression could be written as: weightOperand1 + weightOperand2 + expressionOperand1 + ...
The recursive nature of this type allows complex composition of expressions: ADD(..., MULTIPLY(..., ADD(...)), MAX(...))
(OperatorOrKey)
(string)
($ReadOnlyArray<WeightOperand>)
($ReadOnlyArray<Expression>)
A string from the Operator enum type (ADD, MUPTIPLY, MAX, ...) OR an OperatorConfig key that is an arbitrary string prefixed by "key:"
Type: OperatorOrKey
An arbitrary string describing the level of abstraction / semantic significance / granularity of this expression.
Type: string
An array of WeightOperand leaf nodes that are children of this Expression node. Will be included as operands when the operator is applied.
Type: $ReadOnlyArray<WeightOperand>
An array of Expression nodes that are children of this Expression node. Will be recursively evaluated and then included as operands when the operator is applied.
Type: $ReadOnlyArray<Expression>
A granular contribution that contains the root node of an Expression tree and also has an outgoing array of participants, creating a DAG-like structure.
Responsible for timestamping, containing granular participation details, and linking Expressions to Participants.
(string)
(string)
(string)
(TimestampMs)
(Expression)
($ReadOnlyArray<{id: NodeAddressT, shares: $ReadOnlyArray<WeightOperand>}>)
If the subkey is found, returns the subkey's weight. If the subkey is not found, returns the key's default. Throws if the key has not been set in the configuration.
(WeightOperand)
Name | Description |
---|---|
$0.key any
|
|
$0.subkey any
|
(WeightConfig)
number
Returns true if the subkey exists in the subkeys array of the key. Returns false if the subkey does not exist in the subkeys array. Throws if the key has not been set in the configuration.
(WeightOperand)
Name | Description |
---|---|
$0.key any
|
|
$0.subkey any
|
(WeightConfig)
boolean
Semantically, allows weight configuration of different qualities/characteristics of contributions.
Technically, a once-nested key-value store that maps key-subkey pairs to weights and specifies a default weight at the key-level that can be used when a queried subkey is not found.
A Discord-based example might look like: { "key": "channel", "default": 1, "subkeys": [ { "subkey": "12345678", memo: "props", weight: 3 } ] }
Type: $ReadOnlyArray<{key: string, default: number, subkeys: $ReadOnlyArray<{subkey: string, memo: string?, weight: number}>}>
A key-value store mapping platform-based identifiers to weights.
Type: $ReadOnlyArray<{subkey: string, memo: string?, weight: number}>
A key-value store of configured operators, allowing the configuration of operators within an expression. For example, one might be able to configure that emoji reactions be added or multiplied.
Type: $ReadOnlyArray<{key: string, operator: Operator}>
Wraps the other config types, and defines a time scope via a start date. The end date will be inferred as the next highest start date in an array of Configs.
(string)
(TimestampISO)
(WeightConfig)
(OperatorConfig)
(WeightConfig)
Groups Configs together by target strings that may represent a server ID/endpoint, a repository name, etc.
A note or a human-readable description to make it easier to recognize this config. *
Type: string
Takes a prefixed key and returns the configured operator queried by the non-prefixed key. Throws if the input is not properly prefixed. Throws if the key has not been set in the configuration. Throws if the configured operator is not a valid operator.
(OperatorOrKey)
(Config)
Operator
Utility function for getting the earliest start time of all configs in an array of ConfigsByTarget.
($ReadOnlyArray<ConfigsByTarget>)
TimestampMs
We have a convention of using TimestampMs as our default representation.
However TimestampISO has the benefit of being human readable / writable,
so it's used for serialization and display as well.
We'll validate types at runtime, as there's a fair chance we'll use these
functions to parse data that came from a Flow any
type (like JSON).
Type: number
Creates a TimestampISO from a TimestampMs-like input.
Since much of the previous types have used number
as a type instead of
TimestampMs. Accepting number
will give an easier upgrade path, rather
than a forced refactor across the codebase.
((TimestampMs | number))
TimestampISO
Creates a TimestampMs from a TimestampISO.
((TimestampISO | string))
TimestampMs
Validate that a number is potentially a valid timestamp.
This checks that the number is a finite integer, which avoids some potential numbers that are not valid timestamps.
(number)
TimestampMs
Generic adaptor for persisting a Ledger to some storage backend (e.g. GitHub, local filesystem, a database, etc)
Returns a list of LedgerEvents that have not been persisted to storage yet
(Ledger)
LedgerDiff
Returns a list of LedgerEvents in the persisted ledger that have not been applied to the local ledger.
(Ledger)
LedgerDiff
Persists the local (in-memory) Ledger to the ledger storage. Reloads the remote ledger from storage right before persisting it to minimize the possibility of overwriting remote changes that were not synced to the local ledger and ensure consistency of the ledger events (e.g. no double spends).
A race condition is present in this function: if client A runs reloadLedger and then client B writes to the remote ledger before client A finishes writing, then the changes to the ledger that client B made would be overwritten by the changes from client A. The correctness and consistency of the ledger will still be maintained, its just that client B might experience data loss of whatever events they were trying to sync. To detect if this has occurred, we reload the ledger again after writing the data to ensure the local changes were not overwritten. If they were, we can show an error message to client B with a list of changes that failed to sync.
(...Array<any>)
Promise<ReloadResult>
Reloads the persisted Ledger from storage and replays any local changes on top of any new remote changes, if they exist.
Will return the list of new remote changes as well as a list of local changes that have not been persisted yet. This data is useful for the end user to know:
Promise<ReloadResult>
Returns an array of ledger events that exist in ledger "a" but not in "b". An event is considered equal to another if it has the same uuid.
This will not return any events from "b" that don't exist in "a", so the order of the params matters.
Example 1:
Example 2:
LedgerDiff
get method loads the content specified by path in the GitHub repository.
(string)
relative path to the content.
Promise<Uint8Array>
:
A primary SourceCred API that combines the given inputs into a single WeightedGraph and then runs the CredRank algorithm on it to create a CredGraph containing the cred scores of nodes/participants.
Might mutate the ledger that is passed in.
(CredrankInput)
Promise<CredrankOutput>
Compute CredRank results given a WeightedGraph, a Ledger, and optional parameters.
(WeightedGraph)
(Ledger)
(PersonalAttributions
= []
)
($Shape<MarkovProcessGraphParameters>
= {}
)
($Shape<PagerankOptions>?)
Promise<CredGraph>
This module defines configuration for the Dependencies system, a system which allows a project to mint excess Cred for its dependencies.
To learn about the semantics of the dependencies system, read the module docstring for core/bonusMinting.js
At a high level, this config type allows the instance maintainer to specify identities (usually PROJECT-type identities) to mint extra Cred over time, as a fraction of the baseline instance Cred.
In the future, we'll likely build a UI to manage this config. However, right now it's designed for hand-editability. Also, we really want to be able to ship a default config that adds a SourceCred account (if one doesn't already exist), activates it (if it was just created), and then flows it some Cred.
With that in mind, here's how the config works:
You'll note that the state in the config is a mix of human generated (choosing the name) and automatically maintained (the id). It's something of a weird compromise, but it accomplishes the design objective of having state that's easy for humans to write by hand, but also tracks the vital information by identity id (which is immutable) rather than by name (which is re-nameable).
Note that at present, when the identity in question is re-named, the config must be manually updated to account for the rename. In the future (when the config is automatically maintained) we'll remove this requirement. (Likely we'll stop tracking the identities by name at all in the config; that's an affordance to make the files generatable by hand.)
The ProcessedBonusPolicy is a BonusPolicy which has been transformed so that it matches the abstractions available when we're doing raw cred computation: instead of an address, we track an index into the canonical node order, and rather than arbitrary client-provided periods, we compute the weight for each Interval.
TODO(#1686, @decentralion): Remove this once we switch to CredRank.
Given the weights and types, produce a NodeWeightEvaluator, which assigns a numerical weight to any node.
The weights are interpreted as prefixes, i.e. a given address may match multiple weights. When this is the case, the matching weights are multiplied together. When no weights match, a default weight of 1 is returned.
We currently take the NodeTypes and use them to 'fill in' default type weights if no weight for the type's prefix is explicitly set. This is a legacy affordance; shortly we will remove the NodeTypes and require that the plugins provide the type weights when the Weights object is constructed.
(WeightsT)
NodeWeightEvaluator
Given the weights and the types, produce an EdgeWeightEvaluator, which will assign an EdgeWeight to any edge.
The edge weights are interpreted as prefix matchers, so a single edge may match zero or more EdgeWeights. The weight for the edge will be the product of all matching EdgeWeights (with 1 as the default forwards and backwards weight.)
The types are used to 'fill in' extra type weights. This is a temporary state of affairs; we will change plugins to include the type weights directly in the weights object, so that producing weight evaluators will no longer depend on having plugin declarations on hand.
(WeightsT)
EdgeWeightEvaluator
Create an empty trie backed by the given address module.
(AddressModule<K>)
Add key k
to this trie with value v
. Return this
.
(K)
(V)
this
Get the values in this trie along the path to k
.
More specifically, this method has the following observable
behavior. Let inits
be the list of all prefixes of k
, ordered
by length (shortest to longest). Observe that the length of inits
is n + 1
, where n
is the number of parts of k
; inits
begins
with the empty address and ends with k
itself. Initialize the
result to an empty array. For each prefix p
in inits
, if p
was added to this trie with value v
, then append v
to
result
. Return result
.
(K)
Array<V>
Get the last stored value v
in the path to key k
.
Returns undefined if no value is available.
(K)
V?
This module adds logic for imposing a Cred minting budget on a graph.
Basically, we allow specifiying a budget where nodes matching a particular address may mint at most a fixed amount of Cred per period. Since every plugin writes nodes with a distinct prefix, this may be used to specify plugin-level Cred budgets. The same mechanism could also be used to implement more finely-grained budgets, e.g. for specific node types.
Type:
"WEEKLY"
Given a WeightedGraph and a budget, return a new WeightedGraph which ensures that the budget constraint is satisfied.
Concretely, this means that the weights in the Graph may be reduced, as necessary, in order to bring the total minted Cred within an interval down to the budget's requirements.
(WeightedGraphT)
(Budget)
WeightedGraphT
Given the WeightedGraph and the Budget, returns an array of every {addres, weight} pair where the address needs to be re-weighted in order to satisfy the budget constraint.
(WeightedGraphT)
(Budget)
Reweighting
Given a the time-partitioned graph, the weight evaluator, and a particular entry for the budget, return every {address, weight} pair where the corresponding address needs to be reweighted in order to satisfy this budget entry.
({evaluator: NodeWeightEvaluator, partition: GraphIntervalPartition, entry: BudgetEntry})
Reweighting
Given a WeightedGraph and the reweighting, return a new WeightedGraph which has had its weights updated accordingly, without mutating the original WeightedGraph.
(WeightedGraphT)
(Reweighting)
WeightedGraphT
Given an array of node addresses, return true if any node address is a prefix of another address.
This method runs in O(n^2). This should be fine because it's intended to be run on small arrays (~one per plugin). If this becomes a performance hotpsot, we can write a more performant version.
($ReadOnlyArray<NodeAddressT>)
boolean
Given a MarkovProcessGraph, compute PageRank scores on it.
(MarkovProcessGraph)
($Shape<PagerankOptions>
= {}
)
Promise<CredGraph>
This type resembles the JSON schema for configuring personal attributions, which allows participants to attribute their cred to other participants. This feature should not be used to make cred sellable/transferable, but instead is intended to allow participants to acknowledge that a portion of their credited outputs are directly generated/supported by the labor of others. (e.g. when a contributor has a personal assistant working behind the scenes)
Type: Array<{fromParticipantName: Name, fromParticipantId: IdentityId?, recipients: Array<{toParticipantName: Name, toParticipantId: IdentityId?, proportions: Array<{startDate: TimestampISO, decimalProportion: number}>}>}>
Adds the IdentityIds where only IdentityNames are provided, and updates names and ids to reflect the account's current identity after merging/renaming.
(PersonalAttributionsConfig)
(Ledger)
PersonalAttributionsConfig
Iterates through the provided plugins, runs their contributions
and
identities
processes, and updates the ledger with any new IdentityProposals.
Might mutate the ledger that is passed in.
(ContributionsInput)
(TaskReporter
= new SilentTaskReporter()
)
Promise<ContributionsOutput>
This class is a lightweight utility for reporting task progress to the command line.
elapsed.
finished.
(ConsoleLog?)
(GetTime?)
SilentTaskReporter is a task reporter that collects some information, but does not emit any visible notifications.
It can be used for testing purposes, or as a default TaskReporter for cases where we don't want to default to emitting anything to console.
Rather than emitting any messages or taking timing information, it allows retrieving the sequence of task updates that were sent to the reporter. This makes it easy for test code to verify that the TaskReporter was sent the right sequence of tasks.
Callers can also check what tasks are still active (e.g. to verify that there are no active tasks unfinished at the end of a method.)
ScopedTaskReporter is a higher-order task reporter for generating opaque scopes meant to be passed into child contexts.
In this case, a scope is a log prefix indicating the parent context in which the current task is running.
This allows for reliable filtering and searching on existing tasks
by prefix. Care should be taken to ensure that the same prefix does
not exist in peer task contexts, so far as error handling is concerned,
or a filter may incorrectly catch and finish a still-running task while
error-handling. This risk can be mitigated by only designating prefixes via
a Scoped Task Reporter, as opposed to passing prefixes into the start
and finish
methods manually. For example, this block will always throw:
function f(top: SilentTaskReporter) { top.start("my-prefix: foo"); const scoped = new ScopedTaskReporter(top, "my-prefix"); scoped.start("foo"); // Error: task my-prefix: foo already active }
(TaskReporter)
(string)
A primary SourceCred API that runs the CredEquate algorithm on the given inputs to create ScoredContributions containing info on the cred scores of contributions and the cred earned by participants in each contribution.
(CredequateInput)
CredequateOutput
The input CredGrainView merged with information from the generated scoredContributions
Type: CredGrainView
Scored contributions for the dependencies. 1 per week per dependency.
Type: $ReadOnlyArray<ScoredContribution>
A SourceCred API that generates ScoredContributions to give bonus cred to organizations and projects that the instance depends on or supports.
May mutate the ledger and the dependencies inputs. Will return a new CredGrainView with dependencies included.
(DependenciesInput)
DependenciesOutput
Iterates through the provided plugins, runs their graph
and identities
processes, and updates the ledger with any new IdentityProposals.
Might mutate the ledger that is passed in.
(GraphInput)
($ReadOnlyArray<PluginId>?)
(TaskReporter
= new SilentTaskReporter()
)
Promise<GraphOutput>
A class for composing ReferenceDetectors. Calls ReferenceDetectors in the order they're given in the constructor, returning the first NodeAddressT it encounters.
($ReadOnlyArray<ReferenceDetector>)
A primary SourceCred API that combines the given inputs into a list of grain distributions.
May mutate the ledger that is passed in.
(GrainInput)
Promise<GrainOutput>
Marshall grainInput from a Grain Configuration file for use with executeGrainIntegration function
Promise<GrainIntegrationResults>
This function definition is implemented by Grain Integrations. Grain integrations allow distributions to be executed programmatically beyond the ledger. However, an integration might have some side-effects that require the ledger to be updated, and it therefore has the option of returning a list of of ledger operations. The ledger will update the ledger if accounting is enabled. Otherwise, grain balances will be tracked elsewhere.
Type: function (PayoutDistributions, IntegrationConfig): Promise<PayoutResult>
Return ids sorted by balance.
(DistributionBalances)
$ReadOnlyArray<IdentityId>
Center string in some whitespace for total length {len}.
string
Given some distribution, return the total allocated to id across all allocation policies.
Type: Map<IdentityId, G.Grain>
Given DistributionBalances, return total grain distributed across participants.
(DistributionBalances)
G.Grain
Input type for the analysis API
Output type for the analysis API
(CredAccountData)
(Neo4jOutput?)
A primary SourceCred API that transforms the given inputs into useful data analysis structures.
(AnalysisInput)
Promise<AnalysisOutput>
Iterators that will yield CSV strings. The CSV contents will be batched in groups for scalability. Each group will include headers. These strings can each be written to disk as a .csv file and then used to export the nodes and edges of a CredGraph into a Neo4j database using neo4j-admin.
(any)
(any)
Returns an array of arrays that contains all of the items in the original array parameter, but batched into arrays no larger than the batchSize. Example: batch([1,2,3,4,5], 2) = [[1,2], [3,4], [5]]
($ReadOnlyArray<T>)
(number)
$ReadOnlyArray<$ReadOnlyArray<T>>
Returns an iterator that will stop upon reaching the batchSize, and then can be reused again for more batches. Use the provided hasNext() method to know when there are no more batches available. Example: const result = []; while (iterator.hasNext()) { for (const item of iterator) { // code to process item } // code to finalize batch }
((Iterator<T> | Generator<T, void, void>))
(number)
BatchIterator<T>
This is an Instance implementation that reads and writes using relative paths on the given base URL. The base URL given should end with a trailing slash.
(DataStorage)
Simple read interface for inputs and outputs of the main SourceCred API.
Reads inputs required to run Analysis.
Promise<AnalysisInput>
Reads a weighted graph generated by a previous run of Graph.
(string)
Promise<WeightedGraph>
Reads a CredGrainView generated by a previous run of CredRank.
Promise<CredGrainView>
Simple read/write interface for inputs and outputs of the main SourceCred API.
Extends ReadOnlyInstance
Writes output after running Analysis.
(AnalysisOutput)
Promise<void>
This module contains logic for setting Cred minting budgets over time on a per-plugin basis. As an example, suppose we want to limit the GitHub plugin to mint only 200 Cred per week, and we want the Discord plugin to mint 100 Cred per Week until Jan 1, 2020 and 200 Cred per week thereafter. We could do so with the following config:
(IntervalLength)
({})
This class serves as a simple wrapper for http GET requests using fetch.
(string)
This get method will error if a non-200 or 300-level status was returned.
(string)
Promise<Uint8Array>
Data Storage allows the implementation of a uniform abstraction for I/O
If the value for the key entered cannot be found, an error should be thrown.
(string)
Promise<Uint8Array>
keys should be file-system friendly
Extends DataStorage
The Balanced policy attempts to pay Grain to everyone so that their lifetime Grain payouts are consistent with their lifetime Cred scores.
We recommend use of the Balanced strategy as it takes new information into account-- for example, if a user's contributions earned little Cred in the past, but are now seen as more valuable, the Balanced policy will take this into account and pay them more, to fully appreciate their past contributions.
Type:
"BALANCED"
Allocate a fixed budget of Grain to the users who were "most underpaid".
We consider a user underpaid if they have received a smaller proportion of past earnings than their share of score. They are balanced paid if their proportion of earnings is equal to their score share, and they are overpaid if their proportion of earnings is higher than their share of the score.
We start by imagining a hypothetical world, where the entire grain supply of the project (including this allocation) was allocated according to the current scores. Based on this, we can calculate the "balanced" lifetime earnings for each participant. Usually, some will be "underpaid" (they received less than this amount) and others are "overpaid".
We can sum across all users who were underpaid to find the "total underpayment".
Now that we've calculated each actor's underpayment, and the total underpayment, we divide the allocation's grain budget across users in proportion to their underpayment.
You should use this allocation when you want to divide a fixed budget of grain across participants in a way that aligns long-term payment with total cred scores.
$ReadOnlyArray<GrainReceipt>
The NonnegativeGrain type ensures Grain amount is >= 0, which is particularly useful in the case of policy budgets or grain transfers.
The Immediate policy evenly distributes its Grain budget across users based on their Cred in the most recent interval.
It's used when you want to ensure that everyone gets some consistent reward for participating (even if they may be "overpaid" in a lifetime sense). We recommend using a smaller budget for the Immediate policy.
Type:
"IMMEDIATE"
Split a grain budget in proportion to the cred scores in the most recent time interval, with the option to extend the interval to include the last {numIntervalsLookback} weeks.
$ReadOnlyArray<GrainReceipt>
The Recent policy distributes cred using a time discount factor, weighing recent contributions higher. The policy takes a history of cred scores, progressively discounting past cred scores, and then taking the sum over the discounted scores.
A cred score at time t reads as follows: "The discounted cred c' at a timestep which is n timesteps back from the most recent one is its cred score c multiplied by the discount factor to the nth power."
c' = c * (1 - discount) ** n
Discounts range from 0 to 1, with a higher discount weighing recent contribution higher.
Note that this is a generalization of the Immediate policy, where Immediate is the same as Recent with a full discount, i.e. a discount factor 1 (100%).
Type:
"RECENT"
Split a grain budget based on exponentially weighted recent cred.
$ReadOnlyArray<GrainReceipt>
The Special policy is a power-maintainer tool for directly paying Grain to a target identity. I'm including it because we will use it to create "initialization" payouts to contributors with prior Grain balances in our old ledger.
This has potential for abuse, I don't recommend making it easy to make special payouts from the UI, since it subverts the "Grain comes from Cred" model.
Type:
"SPECIAL"
This is an writable Instance implementation that reads and writes using relative paths.
Extends ReadInstance
(WritableDataStorage)
This is an Instance implementation that reads and writes using relative paths on the local disk.
Extends WriteInstance
(string)
Make a directory, if it doesn't already exist.
(string)
Check if a directory is empty
Will error if a path that resolves to anything other than a directory is provided
(string)
boolean
Disk Storage abstracts away low-level file I/O operations.
(string)
The path
parameter must be relative to the basePath
set at
construction. Any I/O errors when attempting to read contents at the path
will cause an error to be thrown.
(string)
Promise<Uint8Array>
The path
parameter must be relative to the basePath
set at
construction. Any I/O errors when attempting to write contents at the path
will cause an error to be thrown.
(string)
(Uint8Array)
Promise<void>
Normalize the given POSIX path, resolving ".." and "." segments.
When multiple, sequential forward slashes are found, they are replaced by a single forward slash. A trailing forward slash is preserved if present, but not added if absent.
If the path is a zero-length string, "." is returned, representing the current working directory.
A TypeError
is thrown if path
is not a string.
(string)
string
Returns an object mapping owner-name pairs to CLI plugin
declarations; keys are like sourcecred/github
.
(PluginId)
Plugin?
A PluginId uniquely identifies a Plugin.
Each PluginId takes a owner/name
format, as in
sourcecred/github
.
PluginIds are canonically lower-case.
Load and parse a JSON file from DataStorage.
If the file cannot be read, then an error is thrown. If parsing fails, an error is thrown.
Promise<T>
Load and parse a JSON file from DataStorage, with a default to use if the file is not found.
This is intended as a convenience for situations where the user may optionally provide configuration in a json file.
The default must be provided as a function that returns a default, to accommodate situations where the object may be mutable, or where constructing the default may be expensive.
If no file is present at that location, then the default constructor is invoked to create a default value, and that is returned.
If attempting to load the file fails for any reason other than ENOENT or a 404 (e.g. the path actually is a directory), then the error is thrown.
If parsing fails, an error is thrown.
Promise<T>
Read a text file from DataStorage, with a default string value to use if the file is not found. The file is read in the default encoding, UTF-8.
This is intended as a convenience for situations where the user may optionally provide configuration in a non-JSON file saved to disk.
The default must be provided as a function that returns a default, in case constructing the default may be expensive.
If no file is present at that location, then the default constructor is invoked to create a default value, and that is returned.
If attempting to load the file fails for any reason other than ENOENT or a 404 (e.g. the path actually is a directory), then the error is thrown.
Promise<string>
Retrieve previously scraped data for a GitHub repo from cache.
Note: the GithubToken requirement is planned to be removed. See https://github.com/sourcecred/sourcecred/issues/1580
(RepoId)
the GitHub repository to retrieve from cache
(GithubToken)
authentication token to be used for the GitHub API; generate a
token at:
https://github.com/settings/tokens
Name | Description |
---|---|
token.token any
|
|
token.cache any
|
Promise<Repository>
:
a promise that resolves to a JSON object containing the data
scraped from the repository, with data format to be specified
later
Scrape data from a GitHub repo using the GitHub API.
(RepoId)
the GitHub repository to be scraped
(GithubToken)
authentication token to be used for the GitHub API; generate a
token at:
https://github.com/settings/tokens
Name | Description |
---|---|
token.token any
|
|
token.cache any
|
Promise<object>
:
a promise that resolves to a JSON object containing the data
scraped from the repository, with data format to be specified
later
Determine the instant at which our GitHub quota will refresh.
The returned promise may reject with a GithubResponseError
or
string error message.
(GithubToken)
Promise<Date>
Given a resetAt
date response from GitHub, determine the actual
date until which we want to wait. We clamp to a reasonable range and
apply some padding.
Date
A local mirror of a subset of a GraphQL database.
Clients should interact with this module as follows:
Mirror
instance.registerObject
to register a root object of interest.update
to update all transitive dependencies.extract
to retrieve the data in structured form.See the relevant methods for documentation.
(Database)
(Schema.Schema)
({blacklistedIds: $ReadOnlyArray<Schema.ObjectId>?, guessTypename: function (Schema.ObjectId): (Schema.Typename | null)?}?)
Embed the GraphQL schema into the database, initializing it for use as a mirror.
This method should only be invoked once, at construction time.
If the database has already been initialized with the same schema and version, no action is taken and no error is thrown. If the database has been initialized with a different schema or version, the database is left unchanged, and an error is thrown.
A discussion of the database structure follows.
Objects have three kinds of fields: connections, links, and
primitives. The database likewise has a connections
table,
links
table, and primitives
table, each storing corresponding
data for all GraphQL object types.
In more detail:
The connections
table has a row for each (id, fieldname)
pair, where fieldname
is the name of a connection field on the
object with the given ID. This stores metadata about the
connection: its total count, when it was last updated, etc. It
does not store the actual entries in the connection (the nodes
that the connection points to); connection_entries
stores
these.
The links
table has a row for each (id, fieldname)
pair,
where fieldname
is the name of a link field on the object
with the given ID. This simply points to the referenced object.
The primitives
table has a row for each (id, fieldname)
pair, where fieldname
is the name of a (non-ID) primitive
field on the object with the given ID. The value
column holds
the JSON-stringified primitive value: so, for instance, the
JSON value null
is represented as the SQL string 'null',
not SQL NULL, while the JSON string "null" is represented as
the SQL string '"null"'. This is primarily to accommodate
storing booleans: SQLite has no boolean storage class, and we
cannot simply encode true
and false
as 1
and 0
because
we need to be able to distinguish between these respective
values when we read them back out. There are other ways to do
this more efficiently in both space and time (see discussion on
#883 for some options).
These fields are type-specific, and so only exist once a node's typename is set.
We refer to node and primitive data together as "own data", because this is the data that can be queried uniformly for all elements of a type; querying connection data, by contrast, requires the object-specific end cursor.
Nested fields merit additional explanation. The nested field itself
exists on the primitives
table with SQL value NULL, 0, or 1 (as
SQL integers, not strings). As with all other primitives, NULL
indicates that the value has never been fetched. If the value has
been fetched, it is 0 if the nested field itself was null
on the
GraphQL result, or 1 if it was present. This field lets us
distinguish "author: null" from "author: {user: null}".
The "eggs" of a nested field are treated as normal primitive or
link values, whose fieldname is the nested fieldname and egg
fieldname joined by a period. So, if object type Foo
has nested
field bar: Schema.nested({baz: Schema.primitive()})
, then the
primitives
table will include a row with fieldname 'bar.baz'.
Likewise, a row in the links
table might have fieldname
'quux.zod'.
All aforementioned tables are keyed by object ID. Each object also
appears once in the objects
table, which relates its ID,
typename, and last own-data update. Each connection has its own
last-update value, because connections can be updated independently
of each other and of own-data. An object's typename may be NULL
if we have only reached the object through edges of unfaithful
type, in which case we will perform an extra query to definitively
determine fetch the object's type before requesting its own data or
connections.
Note that any object in the database should have entries in the
connections
, links
, and primitives
tables for all relevant
fields, even if the node has never been updated. This is for
convenience of implementation: it means that the first fetch for a
node is the same as subsequent fetches (a simple SQL UPDATE
instead of first requiring an existence check).
A table network_log
logs all GraphQL requests made by this module
and their corresponding responses, as well as the update created
from the response. This is for debugging. For example, if a node in
the database is corrupt in some way, inspecting the network log
will show exactly which queries caused it to enter its broken
state. In theory, it should be possible to replay the network log
to re-create the database state exactly, though no tooling exists
to do so automatically.
Finally, a table meta
is used to store metadata about the mirror
itself. This is used to make sure that the mirror is not loaded
with an incompatible version of the code or schema. It is never
updated after it is first set.
Inform the GraphQL mirror of the existence of an object. The object's ID must be specified. The object's concrete type may also be specified, in which case it must be an OBJECT type in the GraphQL schema.
If the object has previously been registered with the same type, no action is taken and no error is raised. If the object has previously been registered with a different type, an error is thrown, and the database is left unchanged.
({typename: (null | Schema.Typename), id: Schema.ObjectId})
void
As registerObject
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
This internal method also permits registering an object without specifying its typename.
({typename: (null | Schema.Typename), id: Schema.ObjectId})
(function (guess: Schema.Typename): string?)
void
Register an object corresponding to the provided NodeFieldResult
,
if any, returning the object's ID. If the provided value is null
,
no action is taken, no error is thrown, and null
is returned.
As with registerObject
, an error is thrown if an object by the
given ID already exists with a different typename.
This method does not begin or end any transactions. Other methods may call this method as a subroutine in a larger transaction.
See: registerObject
.
(NodeFieldResult)
(function (): string)
(Schema.ObjectId | null)
Find objects and connections that are not known to be up-to-date.
An object's typename is up-to-date if it has ever been fetched from a faithful field reference or a direct typename query.
An object is up-to-date if its own data has been loaded at least as recently as the provided date.
A connection is up-to-date if it has been fetched at least as recently as the provided date, and at the time of fetching there were no more pages.
(Date)
QueryPlan
Create a GraphQL selection set to fetch data corresponding to the given query plan.
The resulting GraphQL should be embedded into a top-level query.
The result of this query is an UpdateResult
.
This function is pure: it does not interact with the database.
(QueryPlan)
Array<Queries.Selection>
Ingest data given by an UpdateResult
. This is porcelain over
_updateConnection
and _updateOwnData
.
See: _findOutdated
.
See: _queryFromPlan
.
(UpdateId)
(UpdateResult)
void
As _updateData
, but do not enter any transactions. Other methods
may call this method as a subroutine in a larger transaction.
(UpdateId)
(UpdateResult)
void
Save the request query and query parameters to the database.
NetworkLogId
Save the network response and response timestamp to the table row corresponding to the request that generated it.
void
Save the UpdateId in the table row corresponding to the network request that generated it.
(NetworkLogId)
(UpdateId)
void
Perform one step of the update loop: find outdated entities, fetch their updates, and feed the results back into the database.
Returns a promise that resolves to true
if any changes were made.
(function ({body: Queries.Body, variables: {}}): Promise<any>)
Promise<boolean>
Update this mirror with new information from a remote GraphQL server.
The postQuery
function should post a GraphQL query to the remote
server and return the data
contents of its response, rejecting if
there are any errors. (Note that this requirement may change
backward-incompatibly in future versions of this module, in the
case that we wish to handle deletions without regenerating all
data.)
The options are as follows:
since
: Fetch all data that is not known to be up-to-date as
of the provided time. For instance, to fetch all objects more
than a day old, use new Date(new Date() - 86400e3)
.
now
: Function to yield the current date, which will be used
as the modification time for any objects or connections updated
in this process. Should probably be () => new Date()
.
connectionPageSize
: Maximum number of entries to fetch in any
given connection. Some providers have a hard limit of 100 on
this value.
connectionLimit
: Maximum number of connections to fetch in a
single query.
See: registerObject
.
See: extract
.
(function ({body: Queries.Body, variables: {}}): Promise<any>)
Promise<void>
Create a GraphQL selection set required to identify an object of the given declared type, which may be either an object type or a union type. This is the minimal required data whenever we find a reference to an object that we want to traverse later.
The fidelity
argument should be the fidelity of the field being
traversed. If it is faithful, the object's typename will be queried
as well; if it is unfaithful, the typename will not be requested.
The resulting GraphQL should be embedded in the context of any node
of the provided type. For instance, _queryShallow("Issue")
returns a selection set that might replace the ?
in any of the
following queries:
repository(owner: "foo", name: "bar") {
issues(first: 1) {
nodes { ? }
}
}
nodes(ids: ["issue#1", "issue#2"]) { ? }
The result of this query has type NodeFieldResult
.
This function is pure: it does not interact with the database.
(Schema.Typename)
(Schema.Fidelity)
Array<Queries.Selection>
Get the current value of the end cursor on a connection, or
undefined
if the object has never been fetched. If no object by
the given ID is known, or the object does not have a connection of
the given name, then an error is thrown.
Note that null
is a valid end cursor and is distinct from
undefined
.
(Schema.ObjectId)
(Schema.Fieldname)
(EndCursor | void)
Create a GraphQL selection set to fetch elements from a collection, specified by its enclosing object type and the connection field name (for instance, "Repository" and "issues").
If the connection has been queried before and you wish to fetch new
elements, use an appropriate end cursor. Use undefined
otherwise.
Note that null
is a valid end cursor and is distinct from
undefined
. Note that these semantics are compatible with the
return value of _getEndCursor
.
If an end cursor for a particular node's connection was specified,
then the resulting GraphQL should be embedded in the context of
that node. For instance, if repository "foo/bar" has ID "baz" and
an end cursor of "c000" on its "issues" connection, then the
GraphQL emitted by _queryConnection("issues", "c000")
might
replace the ?
in the following query:
node(id: "baz") { ? }
If no end cursor was specified, then the resulting GraphQL may be
embedded in the context of any node with a connection of the
appropriate fieldname. For instance, _queryConnection("issues")
emits GraphQL that may replace the ?
in either of the following
queries:
node(id: "baz") { ? } # where "baz" is a repository ID
repository(owner: "foo", name: "bar") { ? }
Note, however, that this query will fetch nodes from the start of the connection. It would be wrong to append these results onto an connection for which we have already fetched data.
The result of this query has type ConnectionFieldResult
.
This function is pure: it does not interact with the database.
See: _getEndCursor
.
See: _updateConnection
.
(Schema.Typename)
(Schema.Fieldname)
((EndCursor | void))
(number)
Array<Queries.Selection>
Ingest new entries in a connection on an existing object.
The connection's last update will be set to the given value, which must be an existing update lest an error be thrown.
If the object does not exist or does not have a connection by the given name, an error will be thrown.
See: _queryConnection
.
See: _createUpdate
.
(UpdateId)
(Schema.ObjectId)
(Schema.Fieldname)
(ConnectionFieldResult)
void
As _updateConnection
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(UpdateId)
(Schema.ObjectId)
(Schema.Fieldname)
(ConnectionFieldResult)
void
Create a GraphQL selection set required to fetch the own-data (primitives and node references) of an object, but not its connection entries. The result depends only on the (concrete) type of the object, not its ID.
The provided typename must correspond to an object type, not a union type; otherwise, an error will be thrown.
The resulting GraphQL can be embedded into the context of any node
with the provided typename. For instance, if there are issues with
IDs "#1" and "#2", then _queryOwnData("Issue")
emits GraphQL
that may replace the ?
in any of the following queries:
repository(owner: "foo", name: "bar") {
issues(first: 1) { ? }
}
nodes(ids: ["#1", "#2") { ... on Issue { ? } }
node(id: "#1") { ... on Issue { ? } }
The result of this query has type E
, where E
is the element
type of OwnDataUpdateResult
. That is, it is an object with shape
that depends on the provided typename: the name of each ID or
primitive field is a key mapping to a primitive value, and the name
of each node field is a key mapping to a NodeFieldResult
.
Additionally, the attribute "__typename" maps to the node's
typename.
This function is pure: it does not interact with the database.
(Schema.Typename)
Array<Queries.Selection>
Ingest own-data (primitive and link) updates for many objects of a
fixed concrete type. Every object in the input list must have the
same __typename
attribute, which must be the name of a valid
object type.
See: _queryOwnData
.
(UpdateId)
(OwnDataUpdateResult)
void
As _updateOwnData
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(UpdateId)
(OwnDataUpdateResult)
void
Create a GraphQL selection set required to fetch the typename of an object. The resulting GraphQL can be embedded in any node context.
The result of this query has type E
, where E
is the element
type of TypenamesUpdateResult
.
This function is pure: it does not interact with the database.
Array<Queries.Selection>
Ingest typenames for many object IDs.
See: _queryTypenames
.
(TypenamesUpdateResult)
void
As _updateTypenames
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(TypenamesUpdateResult)
void
Extract a structured object and all of its transitive dependencies from the database.
The result is an object whose keys are fieldnames, and whose values are:
null
;null
;For instance, the result of extract("issue:1")
might be:
{
__typename: "Issue",
id: "issue:1172",
title: "bug: holding <Space> causes CPU to overheat",
body: "We should fix this immediately.",
author: {
__typename: "User",
id: "user:admin",
login: "admin",
},
comments: [
{
__typename: "IssueComment",
body: "I depend on this behavior; please do not change it.",
author: {
__typename: "User",
id: "user:longtimeuser4",
login: "longtimeuser4",
},
},
{
__typename: "IssueComment",
body: "That's horrifying.",
author: {
__typename: "User",
id: "user:admin",
login: "admin",
},
},
],
timeline: [
{
__typename: "Commit",
messageHeadline: "reinstate CPU warmer (fixes #1172)",
author: {
date: "2001-02-03T04:05:06-07:00",
user: {
__typename: "User",
id: "user:admin",
login: "admin",
}
}
}
],
}
(Here, "title" is a primitive, the "author" field on an issue is a node reference, "comments" is a connection, and the "author" field on a commit is a nested object.)
The returned structure may be circular.
If a node appears more than one time in the result---for instance,
the "user:admin" node above---all instances will refer to the same
object. However, objects are distinct across calls to extract
, so
it is safe to deeply mutate the result of this function.
The provided object ID must correspond to a known object, or an error will be thrown. Furthermore, all transitive dependencies of the object must have been at least partially loaded at some point, or an error will be thrown.
(Schema.ObjectId)
any
Mirrors data from the Discourse API into a local sqlite db.
This class allows us to persist a local copy of data from a Discourse instance. We have it for reasons similar to why we have a GraphQL mirror for GitHub; it allows us to avoid re-doing expensive IO every time we re-load SourceCred. It also gives us robustness in the face of network failures (we can keep however much we downloaded until the fault).
As implemented, the Mirror will never update already-downloaded content, meaning it will not catch edits or deletions. As such, it's advisable to replace the cache periodically (perhaps once a week or month). We may implement automatic cache invalidation in the future.
Each Mirror instance is tied to a particular server. Trying to use a mirror for multiple Discourse servers is not permitted; use separate Mirrors.
Embed the GraphQL schema into the database, initializing it for use as a mirror.
This method should only be invoked once, at construction time.
If the database has already been initialized with the same schema and version, no action is taken and no error is thrown. If the database has been initialized with a different schema or version, the database is left unchanged, and an error is thrown.
A discussion of the database structure follows.
Objects have three kinds of fields: connections, links, and
primitives. The database likewise has a connections
table,
links
table, and primitives
table, each storing corresponding
data for all GraphQL object types.
In more detail:
The connections
table has a row for each (id, fieldname)
pair, where fieldname
is the name of a connection field on the
object with the given ID. This stores metadata about the
connection: its total count, when it was last updated, etc. It
does not store the actual entries in the connection (the nodes
that the connection points to); connection_entries
stores
these.
The links
table has a row for each (id, fieldname)
pair,
where fieldname
is the name of a link field on the object
with the given ID. This simply points to the referenced object.
The primitives
table has a row for each (id, fieldname)
pair, where fieldname
is the name of a (non-ID) primitive
field on the object with the given ID. The value
column holds
the JSON-stringified primitive value: so, for instance, the
JSON value null
is represented as the SQL string 'null',
not SQL NULL, while the JSON string "null" is represented as
the SQL string '"null"'. This is primarily to accommodate
storing booleans: SQLite has no boolean storage class, and we
cannot simply encode true
and false
as 1
and 0
because
we need to be able to distinguish between these respective
values when we read them back out. There are other ways to do
this more efficiently in both space and time (see discussion on
#883 for some options).
These fields are type-specific, and so only exist once a node's typename is set.
We refer to node and primitive data together as "own data", because this is the data that can be queried uniformly for all elements of a type; querying connection data, by contrast, requires the object-specific end cursor.
Nested fields merit additional explanation. The nested field itself
exists on the primitives
table with SQL value NULL, 0, or 1 (as
SQL integers, not strings). As with all other primitives, NULL
indicates that the value has never been fetched. If the value has
been fetched, it is 0 if the nested field itself was null
on the
GraphQL result, or 1 if it was present. This field lets us
distinguish "author: null" from "author: {user: null}".
The "eggs" of a nested field are treated as normal primitive or
link values, whose fieldname is the nested fieldname and egg
fieldname joined by a period. So, if object type Foo
has nested
field bar: Schema.nested({baz: Schema.primitive()})
, then the
primitives
table will include a row with fieldname 'bar.baz'.
Likewise, a row in the links
table might have fieldname
'quux.zod'.
All aforementioned tables are keyed by object ID. Each object also
appears once in the objects
table, which relates its ID,
typename, and last own-data update. Each connection has its own
last-update value, because connections can be updated independently
of each other and of own-data. An object's typename may be NULL
if we have only reached the object through edges of unfaithful
type, in which case we will perform an extra query to definitively
determine fetch the object's type before requesting its own data or
connections.
Note that any object in the database should have entries in the
connections
, links
, and primitives
tables for all relevant
fields, even if the node has never been updated. This is for
convenience of implementation: it means that the first fetch for a
node is the same as subsequent fetches (a simple SQL UPDATE
instead of first requiring an existence check).
A table network_log
logs all GraphQL requests made by this module
and their corresponding responses, as well as the update created
from the response. This is for debugging. For example, if a node in
the database is corrupt in some way, inspecting the network log
will show exactly which queries caused it to enter its broken
state. In theory, it should be possible to replay the network log
to re-create the database state exactly, though no tooling exists
to do so automatically.
Finally, a table meta
is used to store metadata about the mirror
itself. This is used to make sure that the mirror is not loaded
with an incompatible version of the code or schema. It is never
updated after it is first set.
Inform the GraphQL mirror of the existence of an object. The object's ID must be specified. The object's concrete type may also be specified, in which case it must be an OBJECT type in the GraphQL schema.
If the object has previously been registered with the same type, no action is taken and no error is raised. If the object has previously been registered with a different type, an error is thrown, and the database is left unchanged.
({typename: (null | Schema.Typename), id: Schema.ObjectId})
void
As registerObject
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
This internal method also permits registering an object without specifying its typename.
({typename: (null | Schema.Typename), id: Schema.ObjectId})
(function (guess: Schema.Typename): string?)
void
Register an object corresponding to the provided NodeFieldResult
,
if any, returning the object's ID. If the provided value is null
,
no action is taken, no error is thrown, and null
is returned.
As with registerObject
, an error is thrown if an object by the
given ID already exists with a different typename.
This method does not begin or end any transactions. Other methods may call this method as a subroutine in a larger transaction.
See: registerObject
.
(NodeFieldResult)
(function (): string)
(Schema.ObjectId | null)
Find objects and connections that are not known to be up-to-date.
An object's typename is up-to-date if it has ever been fetched from a faithful field reference or a direct typename query.
An object is up-to-date if its own data has been loaded at least as recently as the provided date.
A connection is up-to-date if it has been fetched at least as recently as the provided date, and at the time of fetching there were no more pages.
(Date)
QueryPlan
Create a GraphQL selection set to fetch data corresponding to the given query plan.
The resulting GraphQL should be embedded into a top-level query.
The result of this query is an UpdateResult
.
This function is pure: it does not interact with the database.
(QueryPlan)
Array<Queries.Selection>
Ingest data given by an UpdateResult
. This is porcelain over
_updateConnection
and _updateOwnData
.
See: _findOutdated
.
See: _queryFromPlan
.
(UpdateId)
(UpdateResult)
void
As _updateData
, but do not enter any transactions. Other methods
may call this method as a subroutine in a larger transaction.
(UpdateId)
(UpdateResult)
void
Save the request query and query parameters to the database.
NetworkLogId
Save the network response and response timestamp to the table row corresponding to the request that generated it.
void
Save the UpdateId in the table row corresponding to the network request that generated it.
(NetworkLogId)
(UpdateId)
void
Perform one step of the update loop: find outdated entities, fetch their updates, and feed the results back into the database.
Returns a promise that resolves to true
if any changes were made.
(function ({body: Queries.Body, variables: {}}): Promise<any>)
Promise<boolean>
Update this mirror with new information from a remote GraphQL server.
The postQuery
function should post a GraphQL query to the remote
server and return the data
contents of its response, rejecting if
there are any errors. (Note that this requirement may change
backward-incompatibly in future versions of this module, in the
case that we wish to handle deletions without regenerating all
data.)
The options are as follows:
since
: Fetch all data that is not known to be up-to-date as
of the provided time. For instance, to fetch all objects more
than a day old, use new Date(new Date() - 86400e3)
.
now
: Function to yield the current date, which will be used
as the modification time for any objects or connections updated
in this process. Should probably be () => new Date()
.
connectionPageSize
: Maximum number of entries to fetch in any
given connection. Some providers have a hard limit of 100 on
this value.
connectionLimit
: Maximum number of connections to fetch in a
single query.
See: registerObject
.
See: extract
.
(function ({body: Queries.Body, variables: {}}): Promise<any>)
Promise<void>
Create a GraphQL selection set required to identify an object of the given declared type, which may be either an object type or a union type. This is the minimal required data whenever we find a reference to an object that we want to traverse later.
The fidelity
argument should be the fidelity of the field being
traversed. If it is faithful, the object's typename will be queried
as well; if it is unfaithful, the typename will not be requested.
The resulting GraphQL should be embedded in the context of any node
of the provided type. For instance, _queryShallow("Issue")
returns a selection set that might replace the ?
in any of the
following queries:
repository(owner: "foo", name: "bar") {
issues(first: 1) {
nodes { ? }
}
}
nodes(ids: ["issue#1", "issue#2"]) { ? }
The result of this query has type NodeFieldResult
.
This function is pure: it does not interact with the database.
(Schema.Typename)
(Schema.Fidelity)
Array<Queries.Selection>
Get the current value of the end cursor on a connection, or
undefined
if the object has never been fetched. If no object by
the given ID is known, or the object does not have a connection of
the given name, then an error is thrown.
Note that null
is a valid end cursor and is distinct from
undefined
.
(Schema.ObjectId)
(Schema.Fieldname)
(EndCursor | void)
Create a GraphQL selection set to fetch elements from a collection, specified by its enclosing object type and the connection field name (for instance, "Repository" and "issues").
If the connection has been queried before and you wish to fetch new
elements, use an appropriate end cursor. Use undefined
otherwise.
Note that null
is a valid end cursor and is distinct from
undefined
. Note that these semantics are compatible with the
return value of _getEndCursor
.
If an end cursor for a particular node's connection was specified,
then the resulting GraphQL should be embedded in the context of
that node. For instance, if repository "foo/bar" has ID "baz" and
an end cursor of "c000" on its "issues" connection, then the
GraphQL emitted by _queryConnection("issues", "c000")
might
replace the ?
in the following query:
node(id: "baz") { ? }
If no end cursor was specified, then the resulting GraphQL may be
embedded in the context of any node with a connection of the
appropriate fieldname. For instance, _queryConnection("issues")
emits GraphQL that may replace the ?
in either of the following
queries:
node(id: "baz") { ? } # where "baz" is a repository ID
repository(owner: "foo", name: "bar") { ? }
Note, however, that this query will fetch nodes from the start of the connection. It would be wrong to append these results onto an connection for which we have already fetched data.
The result of this query has type ConnectionFieldResult
.
This function is pure: it does not interact with the database.
See: _getEndCursor
.
See: _updateConnection
.
(Schema.Typename)
(Schema.Fieldname)
((EndCursor | void))
(number)
Array<Queries.Selection>
Ingest new entries in a connection on an existing object.
The connection's last update will be set to the given value, which must be an existing update lest an error be thrown.
If the object does not exist or does not have a connection by the given name, an error will be thrown.
See: _queryConnection
.
See: _createUpdate
.
(UpdateId)
(Schema.ObjectId)
(Schema.Fieldname)
(ConnectionFieldResult)
void
As _updateConnection
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(UpdateId)
(Schema.ObjectId)
(Schema.Fieldname)
(ConnectionFieldResult)
void
Create a GraphQL selection set required to fetch the own-data (primitives and node references) of an object, but not its connection entries. The result depends only on the (concrete) type of the object, not its ID.
The provided typename must correspond to an object type, not a union type; otherwise, an error will be thrown.
The resulting GraphQL can be embedded into the context of any node
with the provided typename. For instance, if there are issues with
IDs "#1" and "#2", then _queryOwnData("Issue")
emits GraphQL
that may replace the ?
in any of the following queries:
repository(owner: "foo", name: "bar") {
issues(first: 1) { ? }
}
nodes(ids: ["#1", "#2") { ... on Issue { ? } }
node(id: "#1") { ... on Issue { ? } }
The result of this query has type E
, where E
is the element
type of OwnDataUpdateResult
. That is, it is an object with shape
that depends on the provided typename: the name of each ID or
primitive field is a key mapping to a primitive value, and the name
of each node field is a key mapping to a NodeFieldResult
.
Additionally, the attribute "__typename" maps to the node's
typename.
This function is pure: it does not interact with the database.
(Schema.Typename)
Array<Queries.Selection>
Ingest own-data (primitive and link) updates for many objects of a
fixed concrete type. Every object in the input list must have the
same __typename
attribute, which must be the name of a valid
object type.
See: _queryOwnData
.
(UpdateId)
(OwnDataUpdateResult)
void
As _updateOwnData
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(UpdateId)
(OwnDataUpdateResult)
void
Create a GraphQL selection set required to fetch the typename of an object. The resulting GraphQL can be embedded in any node context.
The result of this query has type E
, where E
is the element
type of TypenamesUpdateResult
.
This function is pure: it does not interact with the database.
Array<Queries.Selection>
Ingest typenames for many object IDs.
See: _queryTypenames
.
(TypenamesUpdateResult)
void
As _updateTypenames
, but do not enter any transactions. Other
methods may call this method as a subroutine in a larger
transaction.
(TypenamesUpdateResult)
void
Extract a structured object and all of its transitive dependencies from the database.
The result is an object whose keys are fieldnames, and whose values are:
null
;null
;For instance, the result of extract("issue:1")
might be:
{
__typename: "Issue",
id: "issue:1172",
title: "bug: holding <Space> causes CPU to overheat",
body: "We should fix this immediately.",
author: {
__typename: "User",
id: "user:admin",
login: "admin",
},
comments: [
{
__typename: "IssueComment",
body: "I depend on this behavior; please do not change it.",
author: {
__typename: "User",
id: "user:longtimeuser4",
login: "longtimeuser4",
},
},
{
__typename: "IssueComment",
body: "That's horrifying.",
author: {
__typename: "User",
id: "user:admin",
login: "admin",
},
},
],
timeline: [
{
__typename: "Commit",
messageHeadline: "reinstate CPU warmer (fixes #1172)",
author: {
date: "2001-02-03T04:05:06-07:00",
user: {
__typename: "User",
id: "user:admin",
login: "admin",
}
}
}
],
}
(Here, "title" is a primitive, the "author" field on an issue is a node reference, "comments" is a connection, and the "author" field on a commit is a nested object.)
The returned structure may be circular.
If a node appears more than one time in the result---for instance,
the "user:admin" node above---all instances will refer to the same
object. However, objects are distinct across calls to extract
, so
it is safe to deeply mutate the result of this function.
The provided object ID must correspond to a known object, or an error will be thrown. Furthermore, all transitive dependencies of the object must have been at least partially loaded at some point, or an error will be thrown.
(Schema.ObjectId)
any
Decomposition of a schema, grouping types by their kind (object vs. union) and object fields by their kind (primitive vs. link vs. connection).
All arrays contain elements in arbitrary order.
({})
({})
A set of objects and connections that should be updated.
($ReadOnlyArray<Schema.ObjectId>)
($ReadOnlyArray<{typename: Schema.Typename, id: Schema.ObjectId}>)
($ReadOnlyArray<{objectTypename: Schema.Typename, objectId: Schema.ObjectId, fieldname: Schema.Fieldname, endCursor: (EndCursor | void)}>)
An endCursor
of a GraphQL pageInfo
object, denoting where the
cursor should continue reading the next page. This is null
when the
cursor is at the beginning of the connection (i.e., when the
connection is empty, or when first: 0
is provided).
Type: (string | null)
Result describing only the typename of a set of nodes. Used when we only have references to nodes via unfaithful fields.
Type: $ReadOnlyArray<{__typename: Schema.Typename, id: Schema.ObjectId}>
Result describing own-data for many nodes of a given type. Whether a
value is a PrimitiveResult
or a NodeFieldResult
is determined by
the schema.
This type would be exact but for facebook/flow#2977, et al.
Type: $ReadOnlyArray<{__typename: Schema.Typename, id: Schema.ObjectId}>
Result describing new elements for connections on a single node.
This type would be exact but for facebook/flow#2977, et al.
(Schema.ObjectId)
Result describing all kinds of updates. Each key's prefix determines what type of results the corresponding value represents (see constants below). No field prefix is a prefix of another, so this characterization is complete.
This type would be exact but for facebook/flow#2977, et al.
See: _FIELD_PREFIXES
.
A key of an UpdateResult
has this prefix if and only if the
corresponding value represents TypenamesUpdateResult
s.
A key of an UpdateResult
has this prefix if and only if the
corresponding value represents OwnDataUpdateResult
s.
A key of an UpdateResult
has this prefix if and only if the
corresponding value represents NodeConnectionsUpdateResult
s.
Convert a prepared statement into a JS function that executes that statement and asserts that it makes exactly one change to the database.
The prepared statement must use only named parameters, not positional parameters.
The prepared statement must not return data (e.g., INSERT and UPDATE are okay; SELECT is not).
The statement is not executed inside an additional transaction, so in the case that the assertion fails, the effects of the statement are not rolled back by this function.
This is useful when the statement is like UPDATE ... WHERE id = ?
and it is assumed that id
is a primary key for a record already
exists---if either existence or uniqueness fails, this method will
raise an error quickly instead of leading to a corrupt state.
For example, this code...
const setName: ({|+userId: string, +newName: string|}) => void =
_makeSingleUpdateFunction(
"UPDATE users SET name = :newName WHERE id = :userId"
);
setName({userId: "user:foo", newName: "The Magnificent Foo"});
...will update user:foo
's name, or throw an error if there is no
such user or if multiple users have this ID.
(Statement)
function (Args): void
GraphQL structured query data format.
Main module exports:
build
object, providing a fluent builder APIstringify
object, and particularly stringify.body
multilineLayout
and inlineLayout
Type: Array<Definition>
A strategy for stringifying a sequence of GraphQL language tokens.
Create a layout strategy that lays out text over multiple lines, indenting with the given tab string (such as "\t" or " ").
(string)
LayoutStrategy
Create a layout strategy that lays out all text on one line.
LayoutStrategy
Data types to describe a particular subset of GraphQL schemata. Schemata represented by this module must satisfy these constraints:
id
field of primitive type.Type: string
A derived ID to reference a cache layer.
Derives the CacheId for a RepoId.
Returned CacheId
s will be:
(RepoId)
CacheId
Run a retryable operation until it terminates or exhausts its retry
policy. If attempt
ever rejects, this function also immediately
rejects with the same value.
Promise<Result<T, E>>
Mutate the RelationalView, by replacing all of the post bodies with empty strings. Usage of this method is a convenient hack to save space, as we don't currently use the bodies after the _addReferences step. Also removes commit messages.
Creates a Map<URL, NodeAddressT> for each ReferentEntity in this view. Note: duplicates are accepted within one view. However for any URL, the corresponding N.RawAddress should be the same, or we'll throw an error.
Map<URL, NodeAddressT>
Invoke this function when the GitHub GraphQL schema docs indicate that a connection provides a list of nullable nodes, but we expect them all to always be non-null.
This will drop any null
elements from the provided list, issuing a
warning to stderr if null
s are found.
Array<T>
Parse GitHub references from a Markdown document, such as an issue or comment body. This will include references that span multiple lines (across softbreaks), and exclude references that occur within code blocks.
(string)
Array<ParsedReference>
Extract maximal contiguous blocks of text from a Markdown string, in source-appearance order.
For the purposes of this method, code (of both the inline and block varieties) is not considered text, and will not be included in the output at all. HTML contents are similarly excluded.
Normal text, emphasized/strong text, link text, and image alt text
all count as text and will be included. A block of text is not
required to have the same formatting: e.g., the Markdown document
given by hello *there* [you](https://example.com)
without the
backticks has one contiguous block of text: "hello there you"
.
Softbreaks count as normal text, and render as a single space. Hardbreaks break a contiguous block of text.
Block-level elements, such as paragraphs, lists, and block quotes, break contiguous blocks of text.
See test cases for examples.
(string)
Array<string>
Builds a GithubReferenceDetector using multiple RelationalView. As RelationalView should only be used for one repository at a time, you will commonly want to compose several of them into one GithubReferenceDetector.
Note: duplicates are normally expected. However for any URL, the corresponding NodeAddressT should be the same, or we'll throw an error.
($ReadOnlyArray<RelationalView>)
GithubReferenceDetector
A reference detector which uses a pregenerated Map<URL, NodeAddressT>
as a
lookup table.
Note: this is sensitive to canonicalization issues because it's based on string comparisons. For example:
(Map<URL, NodeAddressT>)
A ReferenceDetector which takes a base ReferenceDetector and applies a translate function to any results.
(ReferenceDetector)
(TranslateFunction)
Validates a token against know formatting. Throws an error if it appears invalid.
Personal access token https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line
Installation access token https://developer.github.com/v3/apps/#create-a-new-installation-token
(string)
GithubToken
The serialized form of the Discord config. If you are editing a config.json file, it should match this type.
TODO: This type is kind of disorganized. It would be cleaner to have all the weight configuration in single optional sub-object, I think. Consider cleaning up before 0.8.0.
Type: $ReadOnlyArray<{guildId: Model.Snowflake, reactionWeightConfig: ReactionWeightConfig?, roleWeightConfig: RoleWeightConfig?, channelWeightConfig: ChannelWeightConfig?, propsChannels: $ReadOnlyArray<Model.Snowflake>?, includeNsfwChannels: boolean?, simplifyGraph: boolean?, beginningDate: TimestampISO?}>
Upgrade from the version on disk to the DiscordConfig.
For now, this allows us to refactor to a cleaner internal type without breaking any existing users. We may need this indefinitely if e.g. we decide to de-serialize the raw JSON into maps (since maps can't be written directly to JSON).
(DiscordConfigJson)
DiscordConfigs
All of the information necessary to add a message to the graph, along with its reactions and its mentions.
(Model.Message)
((Model.GuildMember | null))
($ReadOnlyArray<GraphReaction>)
($ReadOnlyArray<GraphMention>)
(Model.Snowflake)
(string)
(Model.Snowflake?)
Find all of the messages that should go into the graph. This will deliberately ignore messages that have no reactions, since they have no Cred impact and don't need to go into the graph.
(SqliteMirrorRepository)
void
TopicHasLikedPost edges connect a Topic to the posts in that topic that were liked, in proportion to the total like weight of the post in question.
See: https://github.com/sourcecred/sourcecred/issues/1896
Parse the links from a Discourse post's cookedHtml, generating an array of UrlStrings. All of the UrlStrings will contain the full server URL (i.e. relative references are made absolute). The serverUrl is required so that we can do this.
Array<UrlString>
Tags can be configured to confer specific like-weight multipliers when added to a Topic. If a tag does not have a configured weight, the defaultWeight is applied. An example configuration might look like:
"weights": {
"defaultTagWeight": 1,
"tagWeights": {
"foo": 0,
"bar": 1.25,
"baz": 2
}
// categoryWeight configs...
}
where foo and bar are the names of tags used in discourse.
When multiple tags are assigned to a topic, their weights are multiplied together to yield a total tag Weight multiplier. In our example configuration, if both foo and bar are added to a topic, likes on posts in the topic will have a weight of 0, (0 * 1.25 = 0), which means that no cred will be minted by those likes.
If "bar" and "baz" are both added to another topic, the likes on all posts in that topic will carry a weight of 2.5 (1.25 * 2 = 2.5), which means that 2.5x as much cred will be minted by those likes.
Type: NodeWeight
Categories can be configured to confer a specific like-weight multiplier when added to a Topic. If a category does not have a configured weight, the defaultWeight is applied. An example configuration might look like:
weights: {
"defaultCategoryWeight": 1,
"categoryWeights": {
"5": 0,
"36": 1.25
}
// tagWeight configs...
}
where "5" and "36" are the categoryIds in discourse.
An easy way to find the categoryId for a given category is to browse to the categories section in discourse (e.g. https://discourse.sourcecred.io/categories). Then mousing over or clicking on a category will bring you to a url that has the shape https://exampleUrl.com/c// Clicking on the community category in sourcecred navigates to https://discourse.sourcecred.io/c/community/26 for example, where the categoryId is 26
Type: NodeWeight
An interface for reading the local Discourse data.
Retrieve every Post available.
The order is unspecified.
$ReadOnlyArray<Post>
Get usernames for all users.
The order is unspecified.
$ReadOnlyArray<User>
Gets all of the like actions in the history.
$ReadOnlyArray<LikeAction>
Gets a Post by ID.
(PostId)
Post?
The timestamp of the last time we've loaded category definition topics. Used for determining whether we should reload them during an update.
Type: number
The most recent bumpedMs timestamp of all topics. Used for determining what the most recent topic changes we have stored in our local mirror, and which we should fetch from the API during update.
Type: number
For the given topic ID, retrieves the bumpedMs value. Returns null, when the topic wasn't found.
(TopicId)
(number | null)
Finds the SyncHeads values, used as input to skip already up-to-date content when mirroring.
SyncHeads
Idempotent insert/replace of a Topic, including all it's Posts.
Note: this will insert new posts, update existing posts and delete old posts. As these are separate queries, we use a transaction here.
(Topic)
($ReadOnlyArray<Post>)
void
Bumps the definitionCheckMs (from SyncHeads) to the provided timestamp.
(TimestampMs)
void
Class for retrieving data from the Discourse API.
The Discourse API implements the JSON endpoints for all functionality of the actual site. As such, it tends to return a lot of information that we don't care about (in contrast to a GraphQL API which would give us only what we ask for). As such, we implement a simple interface over it, which both abstracts over calling the API, and does some post-processing on the results to simplify it to data that is relevant for us.
The "view" received from the Discourse API when getting a topic by ID.
This filters some relevant data like bumpedMs, and the type separation makes this distinction clear.
(TopicId)
(CategoryId)
($ReadOnlyArray<Tag>)
(string)
(TimestampMs)
(string)
The "latest" format Topic from the Discourse API when getting a list of sorted topics.
This filters relevant data like authorUsername, and the type separation makes this distinction clear.
(TopicId)
(CategoryId)
($ReadOnlyArray<Tag>)
(string)
(TimestampMs)
(number)
A complete Topic object.
(any)
(any)
Interface over the external Discourse API, structured to suit our particular needs. We have an interface (as opposed to just an implementation) to enable easy mocking and testing.
Fetches Topics that have been bumped to a higher timestamp than sinceMs
.
Note: this will not be able to find "about-x-category" category definition topics. due to a hard-coded filter in the API. https://github.com/discourse/discourse/blob/594925b8965a26c512665371092fec3383320b58/app/controllers/list_controller.rb#L66
Use categoryDefinitionTopicIds() to find those topics.
(number)
Promise<Array<TopicLatest>>
Parses a "latest" topic.
A "latest" topic, is a topic as returned by the /latest.json API call, and has a distinct assumptions:
usernamesById map used to resolve these IDs to usernames.
(any)
TopicLatest
Discourse ReferenceDetector detector that relies on database lookups.
(ReadRepository)
An intermediate representation of an Initiative.
This makes the assumption a Champion cannot fail in championing. Instead of a success status, they should be removed if unsuccessful.
There is also no timestamp for completion or each edge. It should be inferred from the node timestamps instead. We can support accurate edge timestamps by interpreting wiki histories. However the additional complexity and requirements put on the tracker don't seem worthwhile right now. Especially because cred can flow even before bounties are released. See https://discourse.sourcecred.io/t/write-the-initiatives-plugin/269/6
Represents a source of Initiatives.
Gets an array of all Initiatives in this repository.
$ReadOnlyArray<Initiative>
Represents an "inline contribution" node. They're called entries and named by type: contribution entry, reference entry, dependency entry. The generalization of this is a node entry.
Type:
("DEPENDENCY"
| "REFERENCE"
| "CONTRIBUTION"
)
Takes a NodeEntryJson and normalizes it to a NodeEntry.
Will throw when required fields are missing. Otherwise handles default values and converting ISO timestamps.
(NodeEntryJson)
(TimestampMs)
NodeEntry
Creates a url-friendly-slug from the title of a NodeEntry. Useful for generating a default key.
Note: keys are not required to meet the formatting rules of this slug, this is mostly for predictability and convenience of NodeAddresses.
(string)
string
Represents a single Initiative using a file as source.
Note: The file name will be used to derive the InitiativeId. So it doesn't make sense to use this outside of the context of an InitiativesDirectory.
Type: InitiativeFileV020
When provided with the initiative NodeAddressT of an InitiativeFile this extracts the URL from it. Or null when the address is not for an InitiativeFile.
(NodeAddressT)
(string | null)
Represents an Initiatives directory.
Initiative directories contain a set of InitiativeFiles in a *.json
pattern.
Where the file name is the ID of that Initiative.
Additionally we require a remoteUrl
for this directory. We expect this directory
to be something you can browse online. This allows us to create a ReferenceDetector.
Opaque because we only want this file's functions to create these load results. However we do allow anyone to consume it's properties.
Loads a given InitiativesDirectory.
(InitiativesDirectory)
Promise<LoadedInitiativesDirectory>
A type which supports multiple ways of defining what edges an Initiative has. Currently includes reference detected URLs and NodeEntries. This is the normalized variant of EdgeSpecJson.
($ReadOnlyArray<URL>)
($ReadOnlyArray<NodeEntry>)
Takes an EdgeSpecJson and normalizes it to an EdgeSpec.
Will throw when required fields are missing or duplicate keys are found. Otherwise handles default values and converting ISO timestamps. Note: we allow the EdgeSpecJson to be undefined to easily support omitting edges entirely, while still normalizing to an EdgeSpec.
(EdgeSpecJson?)
(TimestampMs)
EdgeSpec
A separate function to validate an EdgeSpec after it's normalized. Normally you don't need to invoke this directly.
(EdgeSpec)
EdgeSpec
Find the NodeEntries which have a duplicate key.
($ReadOnlyArray<NodeEntry>)
Set<NodeEntry>
Finds elements in the array which are included twice or more. Uses a === comparison, not deep equality.
($ReadOnlyArray<T>)
Set<T>