DEVELOPMENT
We hope everyone had a fantastic August because the development team sure did. After 10 long months, we are ecstatic to announce that we have successfully finished our Quorum Balancing MVP. Having this development complete means that instead of having to trust individual validators, the trust will be based on smaller groups of validators that can be grouped in various ways. This is a massive achievement for the team because this is something that helps us get one step closer to creating the world’s first truly decentralized platform.
Additionally, the team has continued developing the Stoa SDK/API work that will be needed in order to allow 3rd parties to build on our platform. The following is a summary of last months development activity and items that are currently still under development:
Core Development in August:
Monthly Activity:
Last month we had 52 Agora related pull-requests and 30 active issues. Of which:
- 9 pull requests were opened
- 52 pull requests merged
- 5 new issues
- 25 issues closed
Functionality Developed:
“Invalid pre-image data” triggered on non-connected node #1062
This piece of functionality was developed by Ku Chul during the first part of August. This was a bug fix. The following message shouldn’t show up for an empty, non-connected validator. Having it means there’s a bug in pre-image propagation / the Genesis block.
$ dub build
$ cd test/system/node/2/
$ ../../../../build/agora
[…]
[main(ZbnT) INF] Listening for requests on http://0.0.0.0:2826/
Task terminated with uncaught exception: Request failure to http://node-6:6826 after 5 attempts
Task terminated with uncaught exception: Request failure to http://node-7:7826 after 5 attempts
Task terminated with uncaught exception: Request failure to http://node-4:4826 after 5 attempts
Task terminated with uncaught exception: Request failure to http://node-5:5826 after 5 attempts
Task terminated with uncaught exception: Request failure to http://node-3:3826 after 5 attempts
2020–07–27 09:15:36,758 Info [agora .consensus.ValidatorSet] — Invalid pre-image data: The pre-image has a invalid hash value. Pre-image: { enroll_key: 0xb20da9cfbda971f3f573f55eabcd677feaf12f7948e8994a97cdf9e570799b71631e87bb9ebce0d6a402275adfb6e365fdb72139c18559a10df0e5fe4bae08eb, hash: 0x148006017980424589fc1a7f360f815ef1bcce71a3bd0b581f7a126a57449cb058bba5eca3642cac3e93c15ebd2f2d54d206e6ee3c7fc06b8a59e597d5e02840, distance: 11 }
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1062
Use JSON-Schema for verification of JSON data #55
This piece of functionality was developed by Michael during the first part of August. Originally, only JSON data properties are verified. There was a need to use more standardized methods. When Verifying JSON data, a verification must happen in a standard way.
The following is the definition of done:
- Automatically generate the interface using JSON-Schema.
- Create a function to verify JSON data using the interface.
- Apply a new verification method to all sources.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/55
Add a web page with SDK documentation that will be automatically generated when the recent PR is merged #22
This piece of functionality was developed by both Michael and Henry during the first part of August and its self-explanatory. We needed to create documentation that was easy to access and maintain.
Reference sites:
https://github.com/TypeStrong/typedoc
https://typedoc.org/guides/installation/
Automatically generated document sample:
https://typedoc.org/api/
Please refer to the Github link below for more information:
https://github.com/bpfkorea/boa-sdk-ts/issues/22
Apply DB transactions to the Block storage processing #51
This piece of functionality was developed by Henry during the first part of August. This enhancement was to correct if an error occurred while storing blocks and transactions or enrollments, that do not match the ledger actual data may exist. For example, if there is only block data. In this case, rollback must be done and must be restored in the tables.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/51
Quorums should be regularly re-shuffled #1069
This piece of functionality was developed by Drey during the last part of August. Currently, the quorum sets are only changed when there is a change in the validator set either through the addition of new enrollments or by expiring enrollments. Shuffling should be changed to be a frequent process, for example: shuffle quorums every N blocks where N is a consensus parameter.
Be aware that this will require some support code first. In our unit tests, we have tests such as:
- Set validation cycle to 20
- Generate genesis
- Create 19 blocks
- Start the nodes
The nodes won’t have any preimages on startup except the commitment in the enrollment in the genesis block. If we set the “periodic shuffle” N parameter to 10, then on boot-up the nodes would generate the quorums for block height 0 and then on block height 10. But it’s not possible to test this right now because the preimages are missing for height 10, so this would lead to an assertion failure.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1069
A Validator can get stuck trying to connect to expired/discontinued Validators (test-case) #854
This piece of functionality was developed by Jay during the last part of August.
Consider this scenario:
- A Validator starts with an empty blockchain
- The actual latest block height in the network is 100_800
- The Validator uses its DNS seed to find the network
- The Validator will keep trying to discover the network until it finds the Validators with the public keys as set in the Enrollments in the Genesis block
In one scenario the Validators which enrolled in the Genesis block could be expired. Those Validators could even be permanently down. Therefore the Validator will never reach the end of the discovery phase and will become stuck.
What likely needs to be changed:
- The catch-up phase should be done alongside the network discovery phase.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/854
Ensure that an already-enrolled Validator starts to validate immediately #855
This piece of functionality was developed by Jay during the last part of August. As a validator, it is possible that I would boot up when already enrolled (e.g. if the node is a back-up node for another node).
We should have a test that:
- Starts a network with a few nodes;
- Has one node enrolled that is stopped and wiped clean;
- Wait at least 2 blocks;
- Restart the node, verify that it catches up, and starts to validate again;
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/855
Implement Schnorr signature encoding according to SEC1 v2 (v2.3) #304
See section v2.3 of this paper: https://www.secg.org/sec1-v2.pdf
This piece of functionality was developed by Chris during the last part of August. This enhancement was not urgent but was required before a public release, as we’re dependent on Libsodium’s internal state at the moment.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/304
We need the ability to manage logs that occur during execution. #59
This piece of functionality was developed by Henry during the last part of August. If an exception occurs during a process run, it needs to be possible to recover all unsaved data leaving logs.
Simple requirements:
- It shall have the ability to specify the type of log.
- It shall be possible to set the maximum size of the log file.
- The output of the log can be set to a file and console.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/59
Provide a “base docker build” image #825
This piece of functionality was developed by Daniel during the last part of August. Currently, building the docker image from scratch can be quite slow, as it installs 45 packages, including an upgraded GCC. Since we’re depending on edge, this happens quite often, slowing developers and CI alike.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/825
Properly fix hashmap invalidation issue #1024
This PR implemented a workaround: #1023
This piece of functionality was developed by Daniel during the last part of August. However it is inefficient, we should not use an array for all the list of clients since the array may grow too much (although we could put limitations in place, and remove unused clients).
Here is a test-case showing the issue:
import std.stdio;
import agora.utils.Test;
import agora.common.crypto.Key;
void test () @safe
{
int[PublicKey] map;
map[WK .Keys[0].address] = 10;
map[WK .Keys[1].address] = 20;
map[WK .Keys[2].address] = 30;
map[WK .Keys[3].address] = 40;
map[WK .Keys[4].address] = 50;
size_t count;
foreach (ref key, ref value; map)
{
writeln(“In map (before blowItUp) Key: “, key, “, value: “, value);
++count;
if (count == 3)
blowItUp(map);
writeln(“In map (after blowItUp) Key: “, key, “, value: “, value);
}
}
void blowItUp (ref int[PublicKey] map) @safe
{
for (int v = 6; v < 512; ++v)
{
map[WK .Keys[v].address] = v * 10;
writeln(“Adding: Key: “, WK.Keys[v].address, “, value: “, v * 10);
}
}
unittest
{
test();
}
Notice the corruption here in the printouts:
n map (after blowItUp) Key: GDC22CFFKB4ZNRZUP6EMRIGVZSQEPSNH2CBMWLU5GLGKE36M3KX5YD36, value: 30
In map (before blowItUp) Key: GAEAAAAAAAAAAAGPHEVAWAIAAAAAAAAAAAAAAAAAIAA5SCQBAAAABOWV, value: 10
In map (after blowItUp) Key: GAEAAAAAAAAAAAGPHEVAWAIAAAAAAAAAAAAAAAAAIAA5SCQBAAAABOWV, value: 10
In map (before blowItUp) Key: GABQAAAAAAAAAAFKHEVAWAIAAAAAAAAAAAAAAAAAUD5NQCQBAAAAB2H7, value: 10
In map (after blowItUp) Key: GABQAAAAAAAAAAFKHEVAWAIAAAAAAAAAAAAAAAAAUD5NQCQBAAAAB2H7, value: 10
In map (before blowItUp) Key: GAFAAAAAAAAAAAGEHEVAWAIAAAAAAAAAAAAAAAAAUD65QCQBAAAAAYG6, value: 8
In map (after blowItUp) Key: GAFAAAAAAAAAAAGEHEVAWAIAAAAAAAAAAAAAAAAAUD65QCQBAAAAAYG6, value: 8
In map (before blowItUp) Key: GAFAAAAAAAAAAAGYHEVAWAIAAAAAAAAAAAAAAAAA6AA5SCQBAAAAAMIZ, value: 1919903585
In map (after blowItUp) Key: GAFAAAAAAAAAAAGYHEVAWAIAAAAAAAAAAAAAAAAA6AA5SCQBAAAAAMIZ, value: 1919903585
At first, we suspected it was a call to _aaRehash, however, this is actually the user-facing API for the .rehash() call (__aaRehash will not be called automatically). There is probably an internal call to one of the functions here: https://github.com/dlang/druntime/blob/master/src/rt/aaA.d#L126-L155
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1024
Sqlite Database’s close method is still called from a GC thread even when using ManagedDatabase #1122
This piece of functionality was developed by Daniel during the last part of August. There are at least 2 bugs in the current implementation of ManagedDatabase. These bugs cause unittest failures on the latest stable version of some Linux distributions(Debian and Ubuntu).
Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1122
Improve by using Promise for methods of LedgerStorage #69
This piece of functionality was developed by Michael during the last part of August. When using async and await, we can easily write repeating sentences without using the nested callback. But to take advantage of this, we must define the Promise<return type> as a return function. But the methods of LedgerStorage are not. These are modified as functions that return Promise<return type>. This will provide various choices in the use of these methods.
Example of use of async / await / Promis
async function putAllBlockData (ledger_storage: LedgerStorage,
callback: (err: Error | null) => void)
{
// Use Promise to record block data synchronously.
let save = (storage: LedgerStorage, block: any) =>
{
return new Promise<boolean>((resolve, reject) => {
storage.putBlocks(block, ()=>{resolve(true);}, (error)=>{reject(error);});
});
};
for (let elem of sample_data)
{
try
{
await save(ledger_storage, elem);
}
catch (error)
{
callback(error);
return;
}
}
callback(null);
}
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/69
Separate hashing function from class `Hash` #79
This piece of functionality was developed by Michael during the last part of August. The class simplifies having a binary data of hash results of the hash.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/79
Implement the ability to request and recover missing blocks from Agora #45
This piece of functionality was developed by Michael during the last part of August. In order for Stoa to handle the blocks normally, the blocks must be entered exactly in order of their height.
The difference between the height of the block received now and the height of the previously processed block should always be 1.
If this value is 2, it can be inferred that one block is missing.
I created an issue to add the ability to automatically recover when a missing block occurs.
To resolve this, proceed according to the following two procedures:
1. Implement serialization and deserialization functions for all objects included in the Agora block.
2. Implement practical functions necessary for recovering block data.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/45
Implement /preimage_received endpoint #52
This following API was developed by Henry during the last part of August.
API to implement:
POST /preimage_received
The node periodically pushes the preimage.
Stoa does receive the preimage and store it in the Stoa database.
The preimage saves only the last preimage close to cycle_length for each PreImageCycle.
This can enhancement the preimage Infomation of /getValidator and /getValidators.
JSON Type of preimageInfo
{
“enroll_key”:”0x000000000019d6689c085ae”,
“hash”:”0x4a5e1e4baab89f3a32518a88″,
“distance”:6
}
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/52
Stoa has been blocked by CORS policy #46
This following functionality was developed by Henry during the last part of August. When accessing Stoa through a browser, it is not possible to request normally due to the CORS policy.
Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/46
Ongoing Validator Development:
- Implement a per-node request queue for gossiping #1149
- Implement secret key and seed for signature #28
- Upgrade Homebrew LDC to v1.23.0 #39
- Preimage revealing unittest is not actually testing preimage revealing #1071
- Remove `makeChainedTransactions` #1107
- Investigate upstream Vibe.d issue: Handle leak #1136
- /block_externalized endpoint does not need to be disclosed to the public. #76
- Redefines the type of data as same as the block structure of the agora #81
- Add the hash value of the current block header to the database. #84
- Transition away from creating a block based on the number of transactions in the pool #1004
- Node configuration and block storage consistency is not checked #1006
- Ledger routines return boolean status codes but also throw #1083
- Error code is returned when unittest is performed on Windows #820
- Leader selection needs to be improved #1079
- Implement Schnorr-based signature of the block #365
- Implement functions such as hashFull and hashPart in Agora #83
- When storing block data in a database, it is necessary to save storage space. #60
- Implement a lock on the caching directory #829
- Spurious Github CI failures #882
- Implement a VirtualClock to make tests more efficient and reliable #389
- fixing Log.d that can cause segfaults under special circumstances #1128
MARKETING
Summer Getaway Campaign
Our lives have changed due to the 1st and 2nd wave of COVID19 pandemic, which resulted in social distancing and spreading of remote working, and more and more people are getting tired of them. Meanwhile, the summer has arrived, and the people’s hearts became more disturbed with the memory of past days when we could travel freely.
However, since free travel is what we can not enjoy indefinitely, BOSAGORA held a campaign through which community members could share ‘dream place to go for travel’ with each other to recall beautiful memory of the past and envision the brighter future after COVID19 pandemic gets over.
BOSAGORA, which upholds the vision of ‘making a better world’ will unfold various community activities for our community members to overcome COVID era and lead a healthier life.
BOSAGORA campaign with Indian community
BOSAGORA held another campaign to promote BOSAGORA with the Indian community, one of the global communities which produced video contents introducing BOSAGORA early this year.
This campaign was designed to allow more people to experience BOSAGORA contents in Medium and retweet them to their friends to increase the exposure of BOSAGORA to the public. Thanks to such effort, the number of views of contents, Twitter followers, and RT(contents sharing) all increased by multi-hundreds which brought vitality into the community.
UN SDG COVID Citizen Action Campaign
Even after receiving awards for Project Care from the UN, BOSAGORA continues paying attention to various activities of UN SDG contemplating how to contribute to them as a blockchain project. Accordingly, we once again participated in endorsing COVID Citizen Action Campaign by UN SDG for overcoming Corona and the prosperity of humanity.
COVID Citizen Action Campaign consists of 4 calls for the UN and eight requests in each short-, medium-, and long-term recovery phase to participating countries and supporting organizations.
‘Making a better world’ is not what we can achieve in a single event. We will truly contribute to ‘making a better world’ through constant attention to society and the practice of good deeds.
For more information about the campaign, click the following link.
https://bit.ly/2QZ0JlR
Bitboy Contents for August



In his contents for August, Bitboy shed light on the role of BOSAGORA as a ‘project enabler’ incubating promising projects, and our contribution to ‘making a better world’ and realization of UN SDGs. Regarding this, he introduced the recent Jia You campaign, and development progress like TypeScript SDK development.
Watch the video.
https://bit.ly/3lbgsMx
Tech Trends for July: BOSAGORA technology & trend column



Tech Trend is the series of columns of BOSAGORA that looks at the technology and trends in the blockchain industry, and in August, episodes 9 and 10 were published. For more information, click the following link.
#9: BOSAGORA’s own blockchain
https://bit.ly/3byroiS
#10: Introduction of democracy for the revival of cryptocurrency industry
https://bit.ly/3jMx7EK
发表评论