In the bustling ecosystem of blockchain transactions, a crucial part to understanding the underlying dynamics is the analysis of the Ethereum Virtual Machine (EVM) mempool, a holding space for transactions awaiting confirmation. Analyzing the data within this area can offer rich insights into the state of the network at any given time. Leveraging an RPC provider like QuickNode can greatly facilitate this analysis. In this article, we guide you step by step through researching and analyzing EVM mempool data using QuickNode.
Setting Up Your QuickNode Account
QuickNode is the best RPC provider and blockchain development platform on the market. It’s often praised for its comprehensive features, high performance, and dedicated support, making it a preferred choice for many developers and projects seeking reliable access to blockchain networks through its optimized nodes and user-friendly interface.
Head over to Quicknode.com to create a new free account and create a BSC Mainnet Endpoint. If you love the service (which I’m sure you will) use coupon code abqNVcwd to get $20 off if upgrade to the Build or Scale plan.
Also check out their Getting Started guide to ensure your making the most out of your QuickNode Subscription.
Understanding the Basic RPC Calls
To dissect EVM mempool data, acquaint yourself with essential JSON-RPC methods like txpool_content and eth_getTransactionByHash, pivotal to query the mempool effectively.
Getting Started with QuickNode
Step 1: Setting Up Your Development Environment
Equip your development environment with necessary tools like curl or Postman to facilitate HTTP POST requests to the Ethereum network via QuickNode.
Step 2: Making RPC Calls
Fetching Mempool Data with txpool_content
Initiate an HTTP POST request to your QuickNode URL using the txpool_content method to fetch mempool data.
The response structure outlines various details like:
Structure of eth_getTransactionByHash Response:
blockHash: The hash of the block containing the transaction (null if pending).
blockNumber: The block’s number (null if pending).
from: The sender’s address.
gas: The gas limit set by the sender.
gasPrice: The gas price specified by the sender in wei.
hash: The transaction’s unique hash.
input: Encoded data showcasing contract method calls and parameters (for contract interactions).
nonce: The count of transactions sent from the sender’s address.
to: The recipient’s address (null for contract creations).
transactionIndex: The transaction’s position in the block (null if pending).
value: The number of wei transferred.
v, r, s: Cryptographic elements vital for verifying the transaction’s authenticity.
Step 3: Delving into Data Analysis
Analyzing Transaction Patterns
With the retrieved data, delve into transaction patterns like frequency, gas prices, and nonce to gauge the network’s current dynamics and forecast potential trends.
Fee Estimation
Leverage the data to dissect the ongoing fee market. This insight assists in approximating a fitting gas fee for swift transaction confirmations.
Step 4: Furthering Research
Network Congestion Analysis
Evaluate network congestion by scrutinizing the volume and magnitude of transactions in the mempool. This study aids in formulating strategies for optimal transaction timing.
Probing Transaction Strategies
Explore the transaction tactics utilized by various entities, offering a glimpse into how different players optimize their strategies.
Use Cases
Developing Dynamic Fee Algorithms
Harness the analyzed data to craft dynamic fee algorithms that adjust transaction fees based on the mempool’s current state, ensuring efficient transactions even during network congestion.
Crafting Transaction Monitoring Tools
Design tools that monitor transactions, furnishing users with real-time updates on transaction statuses and probable confirmation times.
Enhancing Research and Analysis
Utilize mempool data for deep analysis of network behavior, fostering the creation of optimization strategies and network improvement proposals.
Conclusion
Researching and analyzing EVM mempool data using QuickNode opens doors to a treasure trove of insights into the vibrant landscape of the Ethereum network. This data serves as a cornerstone for the development of advanced monitoring tools, dynamic fee algorithms, and an array of research opportunities. By delving deep into this data, developers and researchers can contribute significantly to the growing and dynamic blockchain ecosystem, fostering innovation and growth.
If you’ve dabbled with Ethereum or any blockchain operating under the Ethereum Virtual Machine (EVM), you might have encountered the term “internal transactions.” These elusive components of the blockchain ecosystem often lead to confusion. Today, we’ll uncover what EVM internal transactions are and how you can analyze them using Remote Procedure Call (RPC) methods.
What Are EVM Internal Transactions?
At a high level, the Ethereum blockchain documents transactions made from one account to another. These are your typical “external transactions” – they’re visible, explicit, and you can find them in any Ethereum block explorer.
However, there’s another layer of transactions happening underneath, the so-called “internal transactions”. These are not explicit transfers of ETH, but are rather operations that occur due to smart contract execution.
For example, suppose you interact with a Decentralized Finance (DeFi) contract to swap one token for another. While the external transaction is your instruction to the contract, the internal transactions could be the contract sending you tokens, deducting fees, interacting with another contract, and so forth.
Analyzing Internal Transactions Using RPC Methods
Ethereum provides a set of RPC methods which developers can use to interact with the network. Some of these methods can be employed to dig deeper into internal transactions:
eth_getTransactionByHash: Begin by retrieving the transaction details using its hash. This provides an overview of the external transaction, including the input data which is essential for decoding smart contract interactions.
trace_replayTransaction: This is a part of the “trace” module, which may not be enabled in all Ethereum nodes. However, it’s invaluable for our purpose. By replaying the transaction, it returns a detailed breakdown of all the operations (including internal transactions) that happened during its execution.
eth_getCode: If you want to delve deeper, this method retrieves the contract code of a given address. This can be vital for understanding what a particular contract is designed to do, especially when observing its internal transactions.
Decoding Contract Input: The input data from our external transaction (retrieved in the first step) can often be decoded using the ABI of the respective contract. This will allow you to understand the specific function called and any parameters passed.
Tools and Platforms: While RPC methods are powerful, there’s a learning curve. Platforms like Etherscan have made it simpler by offering a graphical interface to view internal transactions. However, understanding how to do it manually via RPC gives you flexibility and a deeper understanding.
Key Takeaways:
Internal transactions are operations triggered by contract executions. They are not explicitly recorded as regular transactions but are intrinsic to understanding on-chain activity.
RPC methods provide a gateway to retrieve and analyze these internal transactions. trace_replayTransaction is particularly useful for this purpose.
Familiarizing oneself with these methods and techniques enhances one’s grasp over Ethereum’s intricate workings, ensuring better clarity when developing or interacting with smart contracts.
In a decentralized world powered by smart contracts and blockchain, understanding the underpinnings of these technologies, like internal transactions, becomes invaluable. Embrace these tools and methods to unlock deeper insights into the Ethereum ecosystem.
Do you know how to retrieve data from EVM based chains but are lost with Solana? Consider this post as a primer to Solana for the developer already comfortable with EVM based chains. In this post I discuss various Ethereum Virtual Machine (EVM) based RPC methods and their equivalent counterparts in the Solana blockchain. This post compares and contrasts the methods in terms of their functionality and usage. I give step-by-step guidance and code examples to bridge the gap between the two ecosystems.
Common EVM based methods and their equivalent
eth_getBlockByNumber
The equivalent Solana RPC method for the Ethereum method eth_getBlockByNumber would be the Solana RPC method getBlock. However, it’s important to note that Solana and Ethereum have different structures and terminologies when it comes to their blockchains, so there might be some differences in the data returned and the way you interact with these methods.
In Ethereum, eth_getBlockByNumber allows you to retrieve a block’s information using its block number or block tag. Similarly, in Solana, getBlock allows you to retrieve block information by providing the block’s slot (which is analogous to the block number in Ethereum).
Here’s an example of how you might use the getBlock method in Solana:
# Using solana-clisolanablock<block_slot>
If you’re interacting with the Solana RPC API programmatically, you can use a library like solana-web3.js for JavaScript to call this method:
const web3 =require('@solana/web3.js');const solanaRpcUrl ='QUICKNODE_URL'; // Replace with the quicknode urlconst connection =new web3.Connection(solanaRpcUrl, 'confirmed');asyncfunctiongetSolanaBlock(slot) {const block =await connection.getBlock(slot); console.log(block);}const blockSlot =12345678; // Replace with the desired block slotgetSolanaBlock(blockSlot);
eth_getTransactionReceipt
The Ethereum method eth_getTransactionReceipt retrieves the receipt of a transaction, including information about contract events and logs. In Solana, there isn’t a direct equivalent method because Solana and Ethereum have different transaction models and data structures. However, you can achieve similar functionality using a combination of Solana’s RPC methods and programming features.
To get similar information as eth_getTransactionReceipt in Solana, you would need to follow these steps:
Use Solana’s RPC Methods to Get Transaction Information:
Use the getTransaction Solana RPC method to retrieve transaction details.
Decode Logs and Events:
Solana doesn’t inherently have the concept of logs like Ethereum. Instead, you would need to design your Solana program to emit custom events or data that you’re interested in tracking.
Parse and decode these custom events from the transaction’s transaction message or transaction logs.
Construct a Custom Response:
Based on the decoded events and information, you can construct a custom response object that resembles the information you would get from an Ethereum transaction receipt.
Here’s a simplified example of how you might approach this using JavaScript and Solana’s @solana/web3.js library:
const web3 =require('@solana/web3.js');asyncfunctiongetSolanaTransactionReceipt(transactionSignature) {const solanaRpcUrl ='QUICKNODE_URL'; // Replace with the quicknode urlconst connection =new web3.Connection(solanaRpcUrl, 'confirmed');const transaction =await connection.getTransaction(transactionSignature);if (!transaction) {returnnull; // Transaction not found }// Parse and decode custom events/logs from transaction.messageconst decodedEvents =parseAndDecodeEvents(transaction.message.logs);const receipt = { transactionHash: transactionSignature, blockNumber: transaction.slot, // Solana's equivalent of block number// Other relevant information based on your decoded events decodedEvents, };return receipt;}functionparseAndDecodeEvents(logs) {// Implement logic to parse and decode logs into custom event data// This could involve iterating through logs and extracting data// Refer to Solana's documentation and your program's event structurereturn decodedEvents;}const transactionSignature ='...'; // Replace with the Solana transaction signaturegetSolanaTransactionReceipt(transactionSignature) .then((receipt) => { console.log(receipt); }) .catch((error) => { console.error('Error:', error); });
Remember that Solana’s design and data structures are different from Ethereum’s, so you’ll need to adjust your approach accordingly. The example provided is a high-level guideline, and you’ll need to customize it based on your specific Solana program and use case.
eth_getCode
The Ethereum method eth_getCode is used to retrieve the bytecode of a smart contract deployed on the Ethereum blockchain. The comparable Solana RPC method for this is getAccountInfo, which can be used to retrieve information about an account, including the data associated with it.
Here’s how you can use the Solana getAccountInfo method in comparison to Ethereum’s eth_getCode:
Ethereum eth_getCode:
// Using web3.js for Ethereumconst Web3 =require('web3');const web3 =newWeb3('QUICKNDOE_URL'); // Replace with the quicknode urlasyncfunctiongetEthContractBytecode(address) {const bytecode =await web3.eth.getCode(address);return bytecode;}const contractAddress ='0x...'; // Replace with the Ethereum contract addressgetEthContractBytecode(contractAddress) .then((bytecode) => { console.log(bytecode); }) .catch((error) => { console.error('Error:', error); });
Solana getAccountInfo:
const web3 =require('@solana/web3.js');asyncfunctiongetSolanaContractData(address) {const solanaRpcUrl ='QUICKNODE_URL'; // Replace with the quicknode url const connection =new web3.Connection(solanaRpcUrl, 'confirmed');const publicKey =new web3.PublicKey(address);const accountInfo =await connection.getAccountInfo(publicKey);if (!accountInfo) {returnnull; // Account not found }const contractData = accountInfo.data;return contractData;}const contractAddress ='...'; // Replace with the Solana contract addressgetSolanaContractData(contractAddress) .then((contractData) => { console.log(contractData); }) .catch((error) => { console.error('Error:', error); });
In the Solana example, getAccountInfo retrieves information about the specified account, and the data field of the account info object contains the bytecode or data associated with the contract. Keep in mind that Solana accounts can hold various types of data, not just bytecode, so the contents of the data field might need further processing based on your specific use case.
Adjust the examples according to your use case and the libraries you’re using. Solana’s design and terminology differ from Ethereum’s, so the way you handle contracts and their data will have some distinctions.
eth_call
The Ethereum method eth_call is used to execute a call to a contract’s function or to retrieve contract state without making an actual transaction on the Ethereum network. The comparable Solana RPC method for this is simulateTransaction, which allows you to simulate a transaction and obtain the result of executing a program (smart contract) on the Solana blockchain.
Here’s how you can use the Solana simulateTransaction method in comparison to Ethereum’s eth_call:
Ethereum eth_call:
// Using web3.js for Ethereumconst Web3 =require('web3');const web3 =newWeb3('QUICKNODE_URL'); // Replace with the quicknode url asyncfunctionethCall(contractAddress, data) {const result =await web3.eth.call({ to: contractAddress, data: data, });return result;}const contractAddress ='0x...'; // Replace with the Ethereum contract addressconst inputData ='0x...'; // Replace with the input data for the function callethCall(contractAddress, inputData) .then((result) => { console.log(result); }) .catch((error) => { console.error('Error:', error); });
Solana simulateTransaction:
const web3 =require('@solana/web3.js');asyncfunctionsimulateSolanaTransaction(sender, programId, data) {const solanaRpcUrl ='QUICKNODE_URL'; // Replace with the quicknode url const connection =new web3.Connection(solanaRpcUrl, 'confirmed');const instruction =new web3.TransactionInstruction({ keys: [{ pubkey: sender, isSigner:true, isWritable:true }], programId, data, });const simulatedTransactionResponse =await connection.simulateTransaction(new web3.Transaction().add(instruction) );const simulatedResult = simulatedTransactionResponse.value;return simulatedResult;}const senderPublicKey =new web3.PublicKey('...'); // Replace with the sender's public keyconst programId =new web3.PublicKey('...'); // Replace with the Solana program IDconst inputData = Buffer.from('...'); // Replace with the input data for the programsimulateSolanaTransaction(senderPublicKey, programId, inputData) .then((simulatedResult) => { console.log(simulatedResult); }) .catch((error) => { console.error('Error:', error); });
In the Solana example, simulateTransaction allows you to simulate executing a program (smart contract) using a transaction instruction. Keep in mind that the setup of Solana programs and transactions is different from Ethereum’s, so you’ll need to adapt your code accordingly. Solana’s architecture and design may require additional considerations and adjustments compared to Ethereum’s EVM-based approach.
eth_getBalance
The Ethereum method eth_getBalance is used to retrieve the balance of a specific Ethereum address. The comparable Solana RPC method for this is getBalance, which allows you to retrieve the balance of a specific Solana wallet address.
Here’s how you can use the Solana getBalance method in comparison to Ethereum’s eth_getBalance:
Ethereum eth_getBalance:
// Using web3.js for Ethereumconst Web3 =require('web3');const web3 =newWeb3('QUICKNODE_URL'); // Replace with the quicknode urlasyncfunctiongetEthBalance(address) {const balance =await web3.eth.getBalance(address);return balance;}const ethAddress ='0x...'; // Replace with the Ethereum addressgetEthBalance(ethAddress) .then((balance) => { console.log(balance); }) .catch((error) => { console.error('Error:', error); });
In both cases, you’re retrieving the balance of a specific address, but keep in mind that the address formats and network-specific details will differ between Ethereum and Solana. Solana uses the Ed25519 public key format for addresses, whereas Ethereum uses the hexadecimal format.
Please replace the placeholders with actual addresses and adapt the code according to the libraries and tools you are using. Solana and Ethereum have different architectures and designs, so the code and approach might vary in certain aspects.
debug_traceBlockByHash
Solana’s architecture and design are quite different from Ethereum’s, and while Ethereum provides the debug_traceBlockByHash method to trace and debug blocks, Solana’s RPC methods and tools might not have a direct equivalent with the same level of detail.
In Ethereum, debug_traceBlockByHash allows for detailed tracing and analysis of the execution and state changes within a block. Solana, on the other hand, primarily focuses on high throughput and performance, which might lead to differences in how debugging and tracing are implemented.
Solana’s RPC methods does not offer a direct equivalent for tracing blocks with the same level of detail as Ethereum’s debug_traceBlockByHash. Solana’s design prioritizes speed and scalability, and its focus on parallel processing might make tracing every detail in a block less practical.
However, for debugging and tracing purposes, you might find the following Solana tools and methods useful:
Transaction Explorer: Solana provides a transaction explorer on its block explorer websites (e.g., Solana Explorer). While it might not offer the same level of detail as Ethereum’s block tracing, it can help you visualize transactions and program execution within blocks.
Solana CLI and Logging: The Solana Command-Line Interface (CLI) provides commands to interact with the Solana network. While it might not provide the same tracing capability as Ethereum’s debugging methods, it does offer tools for monitoring transactions and program logs.
Program Logs: Solana programs can emit logs, and these logs can be retrieved using the getLogs RPC method. These logs can provide insights into the execution of Solana programs.
Developer Community: As Solana’s ecosystem evolves, new tools and techniques might emerge for debugging and tracing. It’s recommended to follow Solana’s official channels and developer community discussions for updates and best practices.
Conclusion
Bridging the conceptual and practical gap between EVM-based blockchains and Solana is paramount for developers aiming to diversify their expertise across these platforms. This post illuminated the equivalencies and disparities between Ethereum and Solana RPC methods, from block retrieval to transaction simulation. While Ethereum’s intricate methods offer depth, Solana, with its emphasis on high throughput and scalability, presents its own set of unique methods tailored to its architecture. For the discerning developer, the nuances in interaction, structure, and design between the two ecosystems should serve as a roadmap, guiding them in extracting maximum utility from both platforms. As the blockchain space continues to evolve, it’s imperative to stay updated, embracing new tools, techniques, and best practices that emerge within these dynamic ecosystems.
In my last blog post Programmatically retrieving blockchain data – Part 1, I share a thorough guide on accessing and analyzing blockchain data programmatically. I emphasize the significance and advantages of this approach, highlighting benefits like real-time insights, transparency, trust, tailored analysis, automated auditing, smart contract oversight, innovation, research, and risk control. I walk readers through a detailed process of extracting transaction data from BSC Mainnet. This includes steps like registering with an RPC provider (where I particularly suggest QuickNode), procuring a BSC Mainnet Endpoint, fetching the most recent block number, sifting through that block for transactions, and then delving deep into the transaction details. Throughout the article, I’ve incorporated specific RPC methods and coding illustrations to facilitate a deeper grasp and practical application for my readers.
In this post I will walk through how to automate the process of getting transaction details using Node.js, a common library called web3, and using QuickNode as an RPC Provider.
Exercise
In this blog post, we’ll delve into a Node.js script that uses Web3 to fetch and display transaction details from a specific Ethereum-like blockchain.
Web3 Node.js Library
The web3.js library in Node.js is a powerful tool designed to facilitate interactions with the Ethereum blockchain. It enables developers to connect to local or remote Ethereum nodes, send transactions, deploy and interface with smart contracts, manage Ethereum accounts, and access blockchain data. With its modular design, web3.js abstracts the complexities of Ethereum, providing a streamlined JavaScript API for building decentralized applications and systems on the Ethereum network.
Source Code
Understanding the code
Let’s break down the provided code step by step:
1) Shebang Line
#!/usr/bin/node
This line specifies the interpreter for the script. It indicates that the script should be run using the Node.js runtime.
2) Importing Web3
const { Web3 } =require('web3');
The code imports the Web3 class from the ‘web3’ package, allowing us to interact with Ethereum-compatible blockchains.
3) Blockchain Node URL
var quicknodeurl ='YOUR_QUICKNODE_URL';
This variable holds the URL of a blockchain node. In this case, it’s a QuickNode URL, which serves as the connection point to the blockchain network.
Here, the providerUrl is set to the previously defined QuickNode URL, and a new instance of the Web3 class is created using this provider URL. This instance enables interaction with the blockchain.
This function takes a transaction hash as a parameter and asynchronously fetches transaction details using the web3.eth.getTransaction() method. It then logs the retrieved transaction details to the console.
This function takes a block number as input and fetches details about transactions within that block using the web3.eth.getBlock() method. It logs the block number and associated transaction hashes to the console. It also iterates through the transaction hashes and fetches details for each transaction using the getTransactionDetails() function.
7) Fetching All Blocks
asyncfunctionfetchAllBlocks() {// ...}
This function fetches the latest block number using web3.eth.getBlockNumber() and iterates through blocks from the latest to the genesis block (block number 0). For each block, it calls the getBlockTransactions() function to retrieve and display transaction details.
8) Executing the Fetching Process
fetchAllBlocks();
Finally, the fetchAllBlocks() function is called to initiate the process of fetching and displaying transaction details for all blocks.
Conclusion
In this blog post, we explored a Node.js script that interacts with an Ethereum-compatible blockchain using the Web3 library. The script fetches transaction details and logs them to the console, providing insights into the blockchain’s transaction history. This code can be a useful starting point for anyone looking to programmatically explore and analyze blockchain data. Whether you’re a developer, researcher, or blockchain enthusiast, this example demonstrates how to harness the power of Web3 and Node.js for blockchain interaction.
Welcome to the first installment of this multi-part series, Programmatically Retrieving Blockchain Data. In this series, I will delve into the exciting world of blockchain data retrieval and explore its various aspects, applications, and advantages. Blockchain technology has revolutionized industries ranging from finance to supply chain management, and understanding how to programmatically access and analyze blockchain data can provide invaluable insights and opportunities. In this first part, i will focus on the benefits of programmatically retrieving blockchain data and why it matters in today’s digital landscape.
Benefits and use-cases for Programmatically Retrieving Blockchain Data
Real-Time Insights – Programmatically accessing blockchain data enables developers, analysts, and researchers to obtain real-time insights into transactions, smart contracts, and network activities. This up-to-the-minute information can empower decision-makers to make informed choices based on the most current data available.
Transparency and Trust – Blockchains are known for their transparency and immutability. By programmatically retrieving data, users can independently verify transactions, ensuring that information has not been tampered with. This level of transparency enhances trust among stakeholders and reduces the need for intermediaries.
Customized Analysis – Blockchain data can be vast and complex. By programmatically querying and analyzing specific aspects of the blockchain, users can extract customized insights tailored to their needs. This flexibility allows for targeted research, fraud detection, and trend identification.
Automated Auditing – Traditional auditing processes can be time-consuming and prone to human error. Programmatically retrieving blockchain data streamlines auditing procedures, making them more efficient and accurate. Auditors can verify transactions and compliance measures with greater ease.
Smart Contract Monitoring – Smart contracts are self-executing agreements with predefined conditions. Programmatic access to blockchain data enables real-time monitoring of smart contracts, ensuring that they function as intended and triggering alerts or actions when specific conditions are met.
Innovation and Research – Developers can leverage programmatically retrieved blockchain data to build new applications, services, and products. Researchers can study transaction patterns, user behaviors, and network dynamics to drive innovation in various industries.
Risk Management – Enterprises can use programmatically retrieved blockchain data to assess risks associated with specific transactions or addresses. This data-driven approach enhances risk management strategies and helps prevent fraudulent activities.
Exercise
In this post, we’ll be programmatically accessing transaction data from BSC Mainnet. A blockchain transaction is a digital record representing the movement of data, assets, or information between participants on a blockchain network, authenticated by cryptographic signatures, assigned a unique identifier (transaction hash), and confirmed through a consensus mechanism, thereby ensuring transparency, security, and immutability of the transaction’s details within the blockchain’s ledger.
We’ll be doing the following steps in this exercise:
Sign up for an RPC provider and obtain a BSC Mainnet Endpoint
Get the latest block number from BSC Mainnet
Parse that block for transaction
Return the transaction data for that transaction
Step 1 – Sign up for an RPC Provider and obtain a BSC Mainnet Endpoint
In order to interact with the blockchain programmatically, you need to access the RPC endpoint of the chain. You could use the shared public endpoint provided by the foundation or you sign up for an endpoint with an RPC Provider. A blockchain RPC provider is a software interface that allows external applications to communicate and interact with a blockchain network by making remote procedure calls to access data and services. Using a paid RPC provider offers enhanced reliability, performance, and dedicated support compared to public endpoints, ensuring consistent and high-quality access to blockchain data and services.
QuickNode
QuickNode is the best RPC provider and blockchain development platform on the market. It’s often praised for its comprehensive features, high performance, and dedicated support, making it a preferred choice for many developers and projects seeking reliable access to blockchain networks through its optimized nodes and user-friendly interface.
Head over to Quicknode.com to create a new free account and create a BSC Mainnet Endpoint. If you love the service (which I’m sure you will) use coupon code abqNVcwd to get $20 off if upgrade to the Build or Scale plan.
Also check out their Getting Started guide to ensure your making the most out of your QuickNode Subscription.
Step 2 – Get the latest block number
Blockchain transactions are individual records of data or value transfers, and they are grouped together in blocks, which are sequentially linked to form a chronological and immutable chain of transactions in a blockchain.
We are going to use a RPC method eth_blockNumber to retrieve the current and most recent block number.
curl $INSERT_QUICKNODE_BSC_ENDPOINT \-X POST \-H "Content-Type: application/json" \--data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}'{"jsonrpc":"2.0","id":1,"result":"0xccb93d"}%
The payload from eth_blockNumber returns a hexadecimal string representing the current block number on the blockchain. This number indicates the position of the latest block in the blockchain’s sequence. In this case, the response payload is "0xccb93d", it corresponds to the decimal block number 13416765.
Step 3 – Parse block the block for transactions
We are going to use the RPC method eth_getBlockByNumber to retrieve detailed information about a specific block on the blockchain based on its block number or block tag. This method will return a comprehensive set of data associated with a particular block, including its hash, timestamp, transaction list, and other relevant details.
The payload from eth_getBlockByNumber returns the following key attributes.
number – The block number (as a hexadecimal string) or null if the block is not found.
hash – The block’s unique hash.
parentHash – The hash of the previous block.
nonce – A proof-of-work consensus algorithm value.
sha3Uncles – A hash of the uncles (stale blocks).
logsBloom – A bloom filter related to logs and transactions.
transactionsRoot – The root of the transaction trie.
stateRoot – The root of the state trie.
receiptsRoot – The root of the receipts trie.
miner – The address of the miner who created the block.
difficulty – The block’s difficulty value.
totalDifficulty – The cumulative difficulty of the chain up to this block.
extraData – Extra data associated with the block.
size – The size of the block in bytes.
gasLimit – The maximum gas allowed in the block.
gasUsed – The total gas used by all transactions in the block.
timestamp – The UNIX timestamp of when the block was mined.
transactions – An array of transaction objects included in the block.
We’ll be examining the first transaction returned from eth_getBlockByNumber call. We’ll be using the eth_getTransactionByHash method to retrieve information about a specific transaction on the Ethereum blockchain by providing its unique transaction hash.
When you send a transaction on the Ethereum network, it gets assigned a unique hash, which serves as its identifier. This hash is derived from the transaction’s content, including details like sender address, recipient address, value, data, gas price, and more. It’s a cryptographic fingerprint of the transaction.
By using the eth_getTransactionByHash method, you can query an Ethereum node for detailed information about a transaction based on its hash. The information returned typically includes:
Block Information – The block number and block hash in which the transaction was included.
Transaction Index – The position of the transaction within the block’s list of transactions.
Sender (From) Address – The Ethereum address that initiated the transaction.
Recipient (To) Address – The Ethereum address that received the transaction (if it’s a contract execution, this may be empty).
Transaction Value – The amount of Ether (ETH) transferred in the transaction.
Gas and Gas Price – The gas used by the transaction and the price per unit of gas.
Input Data – Additional data sent with the transaction (often used for smart contract interactions).
Transaction Hash – The unique identifier of the transaction.
Nonce – A nonce is a counter that prevents replay attacks and helps order transactions from a sender.
V, R, S Values – Cryptographic values related to the transaction’s signature.
This method is particularly useful for retrieving and verifying transaction details, checking the status of a transaction (whether it’s pending, mined, or failed), and for building tools, explorers, or applications that require access to specific transaction information on the Ethereum blockchain.
Closing
Programmatically retrieving blockchain data opens up a realm of possibilities for real-time insights, transparency, customization, and innovation. In this multi-part series, we will explore the technical aspects of accessing blockchain data programmatically, including APIs, libraries, and coding examples. Stay tuned for the upcoming parts, where we will delve deeper into the tools and techniques to harness the power of blockchain data for your projects and endeavors.
The opinions expressed are solely my own and do not express the views or opinions of my employer 😀.
Summary
While in Lisbon at the end of 2022 I heard about the Filecoin Saturn launch and thought this was a pretty cool idea to create an on-chain CDN on-top of IPFS. The idea intrigued me to go through the process to setup a node. My setup notes were pretty involved so I figured I’d share it with community.
What is Filecoin Saturn?
Filecoin Saturn is an open-source, community-run Content Delivery Network (CDN) built on Filecoin.
Saturn is a Web3 CDN in Filecoin’s retrieval market. On one side of the network, websites buy fast, low-cost content delivery. On the other side, Saturn node operators earn Filecoin by fulfilling requests.
Prerequisites
Acquire a server with the recommended specifications
At the time of writing this (January 2023) the specifications are as follows:
Minimum Requirements
Recommended Requirements
CPU with 6 cores
CPU with 12+ cores
32GB RAM
128GB+ RAM
1TB SSD storage
4TB+ SSD Storage
10Gbps upload link
10Gbps+ upload link
Provider selection
This workload has a variable cost for data transfer and is based on the usage of the CDN service. Choosing a cloud provider that has less variable cost is more desirable for positive profit margins.
Baremetal Scenario
Some baremetal providers include 20TB of egress traffic with overages being $1 per TB. In this scenario your bandwidth costs would be $10.
Math( (30-20) * 1 = 10 )
Cloud Scenario
AWS charges $0.09 per GB for egress costs with 100GB included for free monthly. In this scenario, your bandwidth costs would be $2961.
Math( (30,000-100) * 0.09 = 2691 )
Create a Filecoin wallet
In order to receive your payouts via FIL you must own a Filecoin wallet. Filecoin wallets allow you to manage FIL, Filecoin’s native token. Wallets store the private keys that allow you to authorize Filecoin transactions, including paying for storage deals and sending FIL to other accounts.
Non-custodial Filecoin Wallets
File coin supports the following wallet implementations:
Glif Wallet – A lightweight web interface to send and receive Filecoin with a Ledger device (instructions).
Glif Safe – A multisig wallet with a lightweight web interface to send and receive Filecoin with a Ledger device (instructions). Check out the Glif Safe GitHub repo for more information.
Lotus – Includes a command-line wallet that can manage bls, sec1p256k1, multisig wallets, and supports Ledger integration.
Creating a Filecoin Wallet on Coinbase
At the time of writing this post, I’m not super bullish on FIL and as a result will be immediately converting my FIL to another currency. For this reason, I chose to create a Filecoin wallet on Coinbase.
To do so, you simply buy 1 FIL on Coinbase with your currency of choice. Then you click on “Send & Receive” and select FIL – The FIL address is your wallet address.
Coinbase will display your wallet address and QR code for sharing. Here is my QR code:
Seriously though, you might want to keep your wallet address private. All information regarding payouts, transfers, etc are public on the blockchain. Also, this data is commonly indexed and made publicly available via block explorers. Filfox is the popular block explorer for FIlecoin.
Preparing for deployment
From the above section you should have:
A server with the requirements and the ability to login into as root
A Filecoin Wallet
Now it’s time to prepare your deployment environment. In this example, I’ll be using my laptop as my deployment host but you could easily use a bastion host.
Ansible Installation
Ansible is an agent-less automation platform used to configure one or many nodes. I ran the following commands to install ansible:
Next we need to install the speedtest cli by ookla – for this I created an ansible playbook called playbooks/speedtest.yml
----name:Speedtest playbooktags:speedtsthosts:allgather_facts:falsetasks:-name:Download speedtest scriptbecome:yesget_url:url:https://packagecloud.io/install/repositories/ookla/speedtest-cli/script.deb.shdest:/homemode:0777-name:Copy and Execute the scriptcommand:bash /home/script.deb.sh-name:Install speedtestapt:name:speedtest
Run the speedtest ansible playbook to install speed test
Then we need to use speedtest to get a list of servers closest to us
ansible -i inventory.yaml saturn -m shell -a "speedtest -L"moon1 | CHANGED | rc=0>>Closest servers: ID Name Location Country==============================================================================45489 Smart City Telecom Lake Buena Vista, FL United States53543 frantic.link Kissimmee, FL United States42450 NexGen Communications Orlando, FL United States10161 CenturyLink Orlando, FL United States28307 Summit Broadband Orlando, FL United States40227 Flash Fiber Orlando, FL United States48319 Whitesky Communications LLC Orlando, FL United States36808 832communications Orlando, FL United States25760 The Villages The Villages, FL United States17170 Spectrum Tampa, FL United States
Add the additional parameters to your .env file
SATURN_NETWORK="main or test"FIL_WALLET_ADDRESS="WalletAddress"NODE_OPERATOR_EMAIL="YourEmail"SPEEDTEST_SERVER_CONFIG="--server-id=PickIDFromLastCommand"SATURN_HOME="/home/"
I removed the section around hardening the OS in the l1.yaml playbook because I have a playbook specifically around this which. I won’t share my playbook for security reasons but you can easily research one or create your own. Please use your own automation or best practice to harden your system. Below is the edited version of l1.yaml.
----name:Configure playbooktags:confighosts:allgather_facts:truetasks:-name:Add docker GPG keybecome:truebecome_user:rootapt_key:url:https://download.docker.com/linux/{{ ansible_distribution | lower }}/gpg-name:Add docker repository (yum)become:truebecome_user:root# ref - https://www.ansiblepilot.com/articles/install-docker-in-redhat-like-systems-ansible-module-rpm_key-yum_repository-and-yum/ansible.builtin.yum_repository:name:dockerdescription:docker repository to CentOSbaseurl:"https://download.docker.com/linux/centos/$releasever/$basearch/stable"enabled:truegpgcheck:truegpgkey:"https://download.docker.com/linux/centos/gpg"state:presentwhen:ansible_distribution == "CentOS"-name:Add docker repository (apt)become:truebecome_user:rootansible.builtin.apt_repository:filename:dockerrepo:deb [arch=amd64] https://download.docker.com/{{ ansible_system | lower }}/{{ ansible_distribution | lower }} {{ ansible_distribution_release }} stablestate:presentwhen:ansible_distribution in ["Debian", "Ubuntu"]-name:Install aptitudebecome:truebecome_user:rootansible.builtin.package:name:-aptitudeupdate_cache:truewhen:ansible_distribution in ["Debian", "Ubuntu"]-name:Install dependenciesbecome:truebecome_user:rootansible.builtin.package:name:-wget-docker-ce-docker-ce-cli-containerd.io-docker-compose-plugin-python3-pipstate:latestupdate_cache:true-name:Install python packagesbecome:truebecome_user:rootansible.builtin.pip:executable:pip3name:-docker>5.0.0state:present-name:Start dockerbecome:truebecome_user:rootansible.builtin.service:name:dockerenabled:truestate:started-name:Run playbooktags:runhosts:allvars:# this wasn't consistent between runs, so we need to store ithomedir:"{{ ansible_env.HOME }}"saturn_home:"{{ saturn_root if saturn_root is defined else homedir }}"env_file:"{{ lookup('file', '../.env' ) }}"tasks:-name:Copy the .env fileansible.builtin.copy:src:../.envdest:"{{ saturn_home }}"-name:Patch SATURN_HOMEansible.builtin.blockinfile:path:"{{ saturn_home }}/.env"block:SATURN_HOME="{{ saturn_home }}"-name:Get docker-compose.ymlbecome:truebecome_user:rootansible.builtin.get_url:url:https://raw.githubusercontent.com/filecoin-saturn/L1-node/main/docker-compose.ymldest:"{{ saturn_home }}/docker-compose.yml"-name:Ensure we have the right $SATURN_HOME/shared permissionsbecome:truebecome_user:rootansible.builtin.file:path:"{{ saturn_home }}/shared"mode:"0755"state:directory-name:Run docker composebecome:truebecome_user:rootansible.builtin.command:docker compose up -d# ansible.builtin.command: docker compose restart watchtower# ansible.builtin.shell: (docker rm -f saturn-node || true) && (docker rm -f saturn-watchtower) && docker compose up -dargs:chdir:"{{ saturn_home }}"
Once complete, check out the logs on the host for any errors by running the following command “docker logs –follow saturn-node”. You can also see in the logs the id of your node.
I will be posting a retrospective of my performance sometime in February of 2023. If you run into issues and need help during your setup, check out the Filecoin Foundation’s Slack: https://filecoin.io/slack.