Reading:
Monthly Report August 2020: Successfully Finished BOSAGORA Quorum Balancing MVP, etc.
Image

Monthly Report August 2020: Successfully Finished BOSAGORA Quorum Balancing MVP, etc.

by Rooney
2020-09-14

DEVELOPMENT

Image for post

We hope everyone had a fantastic August because the development team sure did. After 10 long months, we are ecstatic to announce that we have successfully finished our Quorum Balancing MVP. Having this development complete means that instead of having to trust individual validators, the trust will be based on smaller groups of validators that can be grouped in various ways. This is a massive achievement for the team because this is something that helps us get one step closer to creating the world’s first truly decentralized platform.

Additionally, the team has continued developing the Stoa SDK/API work that will be needed in order to allow 3rd parties to build on our platform. The following is a summary of last months development activity and items that are currently still under development:

Core Development in August:
Monthly Activity:
Last month we had 52 Agora related pull-requests and 30 active issues. Of which:

  • 9 pull requests were opened
  • 52 pull requests merged
  • 5 new issues
  • 25 issues closed

Functionality Developed:

“Invalid pre-image data” triggered on non-connected node #1062

This piece of functionality was developed by Ku Chul during the first part of August. This was a bug fix. The following message shouldn’t show up for an empty, non-connected validator. Having it means there’s a bug in pre-image propagation / the Genesis block.

$ dub build

$ cd test/system/node/2/

$ ../../../../build/agora

[…]

[main(ZbnT) INF] Listening for requests on http://0.0.0.0:2826/

Task terminated with uncaught exception: Request failure to http://node-6:6826 after 5 attempts

Task terminated with uncaught exception: Request failure to http://node-7:7826 after 5 attempts

Task terminated with uncaught exception: Request failure to http://node-4:4826 after 5 attempts

Task terminated with uncaught exception: Request failure to http://node-5:5826 after 5 attempts

Task terminated with uncaught exception: Request failure to http://node-3:3826 after 5 attempts

2020–07–27 09:15:36,758 Info [agora.consensus.ValidatorSet] — Invalid pre-image data: The pre-image has a invalid hash value. Pre-image: { enroll_key: 0xb20da9cfbda971f3f573f55eabcd677feaf12f7948e8994a97cdf9e570799b71631e87bb9ebce0d6a402275adfb6e365fdb72139c18559a10df0e5fe4bae08eb, hash: 0x148006017980424589fc1a7f360f815ef1bcce71a3bd0b581f7a126a57449cb058bba5eca3642cac3e93c15ebd2f2d54d206e6ee3c7fc06b8a59e597d5e02840, distance: 11 }

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1062

Use JSON-Schema for verification of JSON data #55

Image for post

This piece of functionality was developed by Michael during the first part of August. Originally, only JSON data properties are verified. There was a need to use more standardized methods. When Verifying JSON data, a verification must happen in a standard way.

The following is the definition of done:

  • Automatically generate the interface using JSON-Schema.
  • Create a function to verify JSON data using the interface.
  • Apply a new verification method to all sources.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/55

Add a web page with SDK documentation that will be automatically generated when the recent PR is merged #22

This piece of functionality was developed by both Michael and Henry during the first part of August and its self-explanatory. We needed to create documentation that was easy to access and maintain.

Reference sites:
https://github.com/TypeStrong/typedoc
https://typedoc.org/guides/installation/

Automatically generated document sample:
https://typedoc.org/api/

Please refer to the Github link below for more information:
https://github.com/bpfkorea/boa-sdk-ts/issues/22

Apply DB transactions to the Block storage processing #51

This piece of functionality was developed by Henry during the first part of August. This enhancement was to correct if an error occurred while storing blocks and transactions or enrollments, that do not match the ledger actual data may exist. For example, if there is only block data. In this case, rollback must be done and must be restored in the tables.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/51

Quorums should be regularly re-shuffled #1069

Image for post

This piece of functionality was developed by Drey during the last part of August. Currently, the quorum sets are only changed when there is a change in the validator set either through the addition of new enrollments or by expiring enrollments. Shuffling should be changed to be a frequent process, for example: shuffle quorums every N blocks where N is a consensus parameter.

Be aware that this will require some support code first. In our unit tests, we have tests such as:

  • Set validation cycle to 20
  • Generate genesis
  • Create 19 blocks
  • Start the nodes

The nodes won’t have any preimages on startup except the commitment in the enrollment in the genesis block. If we set the “periodic shuffle” N parameter to 10, then on boot-up the nodes would generate the quorums for block height 0 and then on block height 10. But it’s not possible to test this right now because the preimages are missing for height 10, so this would lead to an assertion failure.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1069

A Validator can get stuck trying to connect to expired/discontinued Validators (test-case) #854

This piece of functionality was developed by Jay during the last part of August.

Consider this scenario:

  • A Validator starts with an empty blockchain
  • The actual latest block height in the network is 100_800
  • The Validator uses its DNS seed to find the network
  • The Validator will keep trying to discover the network until it finds the Validators with the public keys as set in the Enrollments in the Genesis block

In one scenario the Validators which enrolled in the Genesis block could be expired. Those Validators could even be permanently down. Therefore the Validator will never reach the end of the discovery phase and will become stuck.

What likely needs to be changed:

  • The catch-up phase should be done alongside the network discovery phase.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/854

Ensure that an already-enrolled Validator starts to validate immediately #855

This piece of functionality was developed by Jay during the last part of August. As a validator, it is possible that I would boot up when already enrolled (e.g. if the node is a back-up node for another node).

We should have a test that:

  • Starts a network with a few nodes;
  • Has one node enrolled that is stopped and wiped clean;
  • Wait at least 2 blocks;
  • Restart the node, verify that it catches up, and starts to validate again;

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/855

Implement Schnorr signature encoding according to SEC1 v2 (v2.3) #304

See section v2.3 of this paper: https://www.secg.org/sec1-v2.pdf

This piece of functionality was developed by Chris during the last part of August. This enhancement was not urgent but was required before a public release, as we’re dependent on Libsodium’s internal state at the moment.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/304

We need the ability to manage logs that occur during execution. #59

This piece of functionality was developed by Henry during the last part of August. If an exception occurs during a process run, it needs to be possible to recover all unsaved data leaving logs.

Simple requirements:

  • It shall have the ability to specify the type of log.
  • It shall be possible to set the maximum size of the log file.
  • The output of the log can be set to a file and console.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/59

Provide a “base docker build” image #825

This piece of functionality was developed by Daniel during the last part of August. Currently, building the docker image from scratch can be quite slow, as it installs 45 packages, including an upgraded GCC. Since we’re depending on edge, this happens quite often, slowing developers and CI alike.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/825

Properly fix hashmap invalidation issue #1024

This PR implemented a workaround: #1023

This piece of functionality was developed by Daniel during the last part of August. However it is inefficient, we should not use an array for all the list of clients since the array may grow too much (although we could put limitations in place, and remove unused clients).

Here is a test-case showing the issue:

import std.stdio;

import agora.utils.Test;

import agora.common.crypto.Key;

void test () @safe

{

int[PublicKey] map;

map[WK.Keys[0].address] = 10;

map[WK.Keys[1].address] = 20;

map[WK.Keys[2].address] = 30;

map[WK.Keys[3].address] = 40;

map[WK.Keys[4].address] = 50;

size_t count;

foreach (ref key, ref value; map)

{

writeln(“In map (before blowItUp) Key: “, key, “, value: “, value);

++count;

if (count == 3)

blowItUp(map);

writeln(“In map (after blowItUp) Key: “, key, “, value: “, value);

}

}

void blowItUp (ref int[PublicKey] map) @safe

{

for (int v = 6; v < 512; ++v)

{

map[WK.Keys[v].address] = v * 10;

writeln(“Adding: Key: “, WK.Keys[v].address, “, value: “, v * 10);

}

}

unittest

{

test();

}

Notice the corruption here in the printouts:

n map (after blowItUp) Key: GDC22CFFKB4ZNRZUP6EMRIGVZSQEPSNH2CBMWLU5GLGKE36M3KX5YD36, value: 30

In map (before blowItUp) Key: GAEAAAAAAAAAAAGPHEVAWAIAAAAAAAAAAAAAAAAAIAA5SCQBAAAABOWV, value: 10

In map (after blowItUp) Key: GAEAAAAAAAAAAAGPHEVAWAIAAAAAAAAAAAAAAAAAIAA5SCQBAAAABOWV, value: 10

In map (before blowItUp) Key: GABQAAAAAAAAAAFKHEVAWAIAAAAAAAAAAAAAAAAAUD5NQCQBAAAAB2H7, value: 10

In map (after blowItUp) Key: GABQAAAAAAAAAAFKHEVAWAIAAAAAAAAAAAAAAAAAUD5NQCQBAAAAB2H7, value: 10

In map (before blowItUp) Key: GAFAAAAAAAAAAAGEHEVAWAIAAAAAAAAAAAAAAAAAUD65QCQBAAAAAYG6, value: 8

In map (after blowItUp) Key: GAFAAAAAAAAAAAGEHEVAWAIAAAAAAAAAAAAAAAAAUD65QCQBAAAAAYG6, value: 8

In map (before blowItUp) Key: GAFAAAAAAAAAAAGYHEVAWAIAAAAAAAAAAAAAAAAA6AA5SCQBAAAAAMIZ, value: 1919903585

In map (after blowItUp) Key: GAFAAAAAAAAAAAGYHEVAWAIAAAAAAAAAAAAAAAAA6AA5SCQBAAAAAMIZ, value: 1919903585

At first, we suspected it was a call to _aaRehash, however, this is actually the user-facing API for the .rehash() call (__aaRehash will not be called automatically). There is probably an internal call to one of the functions here: https://github.com/dlang/druntime/blob/master/src/rt/aaA.d#L126-L155

Please refer to the Github link below for more information:

https://github.com/bpfkorea/agora/issues/1024

Sqlite Database’s close method is still called from a GC thread even when using ManagedDatabase #1122

This piece of functionality was developed by Daniel during the last part of August. There are at least 2 bugs in the current implementation of ManagedDatabase. These bugs cause unittest failures on the latest stable version of some Linux distributions(Debian and Ubuntu).

Please refer to the Github link below for more information:
https://github.com/bpfkorea/agora/issues/1122

Improve by using Promise for methods of LedgerStorage #69

This piece of functionality was developed by Michael during the last part of August. When using async and await, we can easily write repeating sentences without using the nested callback. But to take advantage of this, we must define the Promise<return type> as a return function. But the methods of LedgerStorage are not. These are modified as functions that return Promise<return type>. This will provide various choices in the use of these methods.

Example of use of async / await / Promis

async function putAllBlockData (ledger_storage: LedgerStorage,

callback: (err: Error | null) => void)

{

// Use Promise to record block data synchronously.

let save = (storage: LedgerStorage, block: any) =>

{

return new Promise<boolean>((resolve, reject) => {

storage.putBlocks(block, ()=>{resolve(true);}, (error)=>{reject(error);});

});

};

for (let elem of sample_data)

{

try

{

await save(ledger_storage, elem);

}

catch (error)

{

callback(error);

return;

}

}

callback(null);

}

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/69

Separate hashing function from class `Hash` #79

This piece of functionality was developed by Michael during the last part of August. The class simplifies having a binary data of hash results of the hash.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/79

Implement the ability to request and recover missing blocks from Agora #45

This piece of functionality was developed by Michael during the last part of August. In order for Stoa to handle the blocks normally, the blocks must be entered exactly in order of their height.

The difference between the height of the block received now and the height of the previously processed block should always be 1.

If this value is 2, it can be inferred that one block is missing.

I created an issue to add the ability to automatically recover when a missing block occurs.

To resolve this, proceed according to the following two procedures:

1. Implement serialization and deserialization functions for all objects included in the Agora block.

2. Implement practical functions necessary for recovering block data.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/45

Implement /preimage_received endpoint #52

This following API was developed by Henry during the last part of August.

API to implement:
POST /preimage_received

The node periodically pushes the preimage.

Stoa does receive the preimage and store it in the Stoa database.

The preimage saves only the last preimage close to cycle_length for each PreImageCycle.

This can enhancement the preimage Infomation of /getValidator and /getValidators.

JSON Type of preimageInfo

{

“enroll_key”:”0x000000000019d6689c085ae”,

“hash”:”0x4a5e1e4baab89f3a32518a88″,

“distance”:6

}

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/52

Stoa has been blocked by CORS policy #46

This following functionality was developed by Henry during the last part of August. When accessing Stoa through a browser, it is not possible to request normally due to the CORS policy.

Please refer to the Github link below for more information:
https://github.com/bpfkorea/stoa/issues/46

Ongoing Validator Development:


MARKETING

Summer Getaway Campaign

Image for post

Our lives have changed due to the 1st and 2nd wave of COVID19 pandemic, which resulted in social distancing and spreading of remote working, and more and more people are getting tired of them. Meanwhile, the summer has arrived, and the people’s hearts became more disturbed with the memory of past days when we could travel freely.

However, since free travel is what we can not enjoy indefinitely, BOSAGORA held a campaign through which community members could share ‘dream place to go for travel’ with each other to recall beautiful memory of the past and envision the brighter future after COVID19 pandemic gets over.

BOSAGORA, which upholds the vision of ‘making a better world’ will unfold various community activities for our community members to overcome COVID era and lead a healthier life.

BOSAGORA campaign with Indian community

Image for post

BOSAGORA held another campaign to promote BOSAGORA with the Indian community, one of the global communities which produced video contents introducing BOSAGORA early this year.

This campaign was designed to allow more people to experience BOSAGORA contents in Medium and retweet them to their friends to increase the exposure of BOSAGORA to the public. Thanks to such effort, the number of views of contents, Twitter followers, and RT(contents sharing) all increased by multi-hundreds which brought vitality into the community.

UN SDG COVID Citizen Action Campaign

Image for post

Even after receiving awards for Project Care from the UN, BOSAGORA continues paying attention to various activities of UN SDG contemplating how to contribute to them as a blockchain project. Accordingly, we once again participated in endorsing COVID Citizen Action Campaign by UN SDG for overcoming Corona and the prosperity of humanity.

COVID Citizen Action Campaign consists of 4 calls for the UN and eight requests in each short-, medium-, and long-term recovery phase to participating countries and supporting organizations.

‘Making a better world’ is not what we can achieve in a single event. We will truly contribute to ‘making a better world’ through constant attention to society and the practice of good deeds.

For more information about the campaign, click the following link.
https://bit.ly/2QZ0JlR

Bitboy Contents for August

Image for post

In his contents for August, Bitboy shed light on the role of BOSAGORA as a ‘project enabler’ incubating promising projects, and our contribution to ‘making a better world’ and realization of UN SDGs. Regarding this, he introduced the recent Jia You campaign, and development progress like TypeScript SDK development.

Watch the video.
https://bit.ly/3lbgsMx

Tech Trends for July: BOSAGORA technology & trend column

Image for post

Tech Trend is the series of columns of BOSAGORA that looks at the technology and trends in the blockchain industry, and in August, episodes 9 and 10 were published. For more information, click the following link.

#9: BOSAGORA’s own blockchain
https://bit.ly/3byroiS

#10: Introduction of democracy for the revival of cryptocurrency industry
https://bit.ly/3jMx7EK


0 Comments

Leave a Reply

Related Stories

2020-05-20

1st Year of BOSAGORA

by
Arrow-up