trustwallet-assets/script/action/binance.ts

144 lines
5.2 KiB
TypeScript
Raw Normal View History

[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
import axios from "axios";
import * as bluebird from "bluebird";
import * as fs from "fs";
import * as path from "path";
import * as chalk from 'chalk';
2020-08-19 20:34:28 +00:00
import * as config from "../config";
import { ActionInterface, CheckStepInterface } from "./interface";
import { getChainAssetsPath } from "../common/repo-structure";
import { Binance } from "../common/blockchains";
import { readDirSync } from "../common/filesystem";
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
import {
getChainAssetLogoPath,
getChainDenylistPath
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
} from "../common/repo-structure";
const binanceChain = "binance"
2020-08-19 20:34:28 +00:00
const binanceUrlTokens2 = config.binanceUrlTokens2;
const binanceUrlTokens8 = config.binanceUrlTokens8;
const binanceUrlTokenAssets = config.binanceUrlTokenAssets;
var cachedAssets = [];
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
async function retrieveBep2AssetList(): Promise<any[]> {
console.log(` Retrieving token asset infos from: ${binanceUrlTokenAssets}`);
const { assetInfoList } = await axios.get(binanceUrlTokenAssets).then(r => r.data);
console.log(` Retrieved ${assetInfoList.length} token asset infos`);
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
return assetInfoList
}
async function retrieveAssets(): Promise<any[]> {
// cache results because of rate limit, used more than once
if (cachedAssets.length == 0) {
console.log(` Retrieving token infos (${binanceUrlTokens2}, ${binanceUrlTokens8})`);
const bep2assets = await axios.get(binanceUrlTokens2);
const bep8assets = await axios.get(binanceUrlTokens8);
cachedAssets = bep2assets.data.concat(bep8assets.data);
}
console.log(` Using ${cachedAssets.length} assets`);
return cachedAssets;
}
export async function retrieveAssetSymbols(): Promise<string[]> {
const assets = await retrieveAssets();
const symbols = assets.map(({ symbol }) => symbol);
return symbols;
}
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
function fetchImage(url) {
return axios.get(url, { responseType: "stream" })
.then(r => r.data)
.catch(err => {
throw `Error fetchImage: ${url} ${err.message}`;
});
}
/// Return: array with images to fetch; {asset, assetImg}
export function findImagesToFetch(assetInfoList: any, denylist: string[]): any[] {
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
let toFetch: any[] = [];
console.log(`Checking for asset images to be fetched`);
assetInfoList.forEach(({asset, assetImg}) => {
process.stdout.write(`.${asset} `);
if (assetImg) {
if (denylist.indexOf(asset) != -1) {
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
console.log();
console.log(`${asset} is denylisted`);
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
} else {
const imagePath = getChainAssetLogoPath(binanceChain, asset);
if (!fs.existsSync(imagePath)) {
console.log(chalk.red(`Missing image: ${asset}`));
toFetch.push({asset, assetImg});
}
}
}
});
console.log();
console.log(`${toFetch.length} asset image(s) to be fetched`);
return toFetch;
}
async function fetchMissingImages(toFetch: any[]): Promise<string[]> {
console.log(`Attempting to fetch ${toFetch.length} asset image(s)`);
let fetchedAssets: string[] = [];
await bluebird.each(toFetch, async ({ asset, assetImg }) => {
if (assetImg) {
const imagePath = getChainAssetLogoPath(binanceChain, asset);
fs.mkdir(path.dirname(imagePath), err => {
if (err && err.code != `EEXIST`) throw err;
});
await fetchImage(assetImg).then(buffer => {
buffer.pipe(fs.createWriteStream(imagePath));
fetchedAssets.push(asset)
console.log(`Fetched image ${asset} ${imagePath} from ${assetImg}`)
});
}
});
console.log();
return fetchedAssets;
}
export class BinanceAction implements ActionInterface {
getName(): string { return "Binance chain"; }
getSanityChecks(): CheckStepInterface[] {
return [
{
getName: () => { return "Binance chain; assets must exist on chain"},
check: async () => {
var error: string = "";
const tokenSymbols = await retrieveAssetSymbols();
const assets = readDirSync(getChainAssetsPath(Binance));
assets.forEach(asset => {
if (!(tokenSymbols.indexOf(asset) >= 0)) {
error += `Asset ${asset} missing on chain\n`;
}
});
console.log(` ${assets.length} assets checked.`);
return error;
}
},
];
}
getConsistencyChecks = null;
sanityFix = null;
consistencyFix = null;
async update(): Promise<void> {
// retrieve missing token images; BEP2 (bep8 not supported)
const bep2InfoList = await retrieveBep2AssetList();
const denylist: string[] = require(getChainDenylistPath(binanceChain));
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
const toFetch = findImagesToFetch(bep2InfoList, denylist);
const fetchedAssets = await fetchMissingImages(toFetch);
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
if (fetchedAssets.length > 0) {
console.log(`Fetched ${fetchedAssets.length} asset(s):`);
fetchedAssets.forEach(asset => console.log(` ${asset}`));
}
[Internal] Refactored fix builds (#2920) * Empty infrastructure for new-housekeeping build. * Move updateBEP action to new new-housekeeping build infra. * Remove old updateBEP. * New-housekeeping-dryrun run. * Include new top-level folder name script-new. * Remove update:bep2 from old daily-run. * Use imports instead of require. * Small refactor for testability. * Organize scripts into subfolders. * iUpdateBEP2: refactor and add tests. * Move formatting validators to new-housekeeping,add new helpers. * Move info and black/whitelist fixing to new-housekeeping. * New fix command. * New 'fix' target; Move ETH checksum fix to new-housekeeping. * Move logo size check and resize to new-housekeeping. * Improved async error handling. * Build renames. * Move (old) BEP2 and CMC update to periodic update build. * Rename (add missing). * Rename builds. * Renames ('fix'). * rename * Invoke new scripts (as well) from period-update. * Move cmc update to new-periodic. * Move tezos validator update to new-periodic. * Missing file. * Leftover. * Cleanup * Rename of unused openseacontracts. * CMC should not be run always. * Break main/fixAndUpdate function into two. * Show diff in build after changes. * Cleanup * Rename, script-old. * Cleanup, remove old fix build definitions. * Renames, remove new- prefix. * CMC mapping update. * Config infrastructure; add binance URL to config. * Add image size parameters to config. * Rename. Co-authored-by: Catenocrypt <catenocrypt@users.noreply.github.com> Co-authored-by: Andrew M <35627271+zachzwei@users.noreply.github.com>
2020-07-29 13:42:51 +00:00
}
}