Summary
The idea behind the automated crypto trading is simply to not watch the markets constantly and miss my regular life. I hated missing out on playing with my young children and feeling anxiety about missing a trade. The platform utilizes a series of python programs and libraries such as SKLearn, Pandas, and Numpy to collect data on various crypto currencies, store it in a database, prepare it, analyze it, make predictions, and use APIs to trade through legitimate crypto brokers.
History
This project originated to trade stocks. The problem was that I was only allowed 3 day trades per week without having an account balance of over $32,000. Not wanting to make mistakes. I changed the format to crypto which could be traded all day, every day, and as many times per day as I felt like.
When I first started trading crypto, I was filled with anxiety. My children wanted to play. The best time to sell always seemed to be when I picked them up from school. I was afraid to go to sleep and would wake up at odd times. I was trying to get a feel for how the market ran and if there were any margins I could exploit. There were. Very often a currency would change value by over 10%. In truth, the cost of buying and selling a currency is very low. You could very well make a profit on 4%.
Planning
Many people were making money mining crypto currency. They were spending thousands of dollars building computers and spending hundreds of extra dollars on electricity for something they may not even come out even on. I didn’t want to have anything to do with any of this. I wanted to use the least amount of power that I could. So I went with a Raspberry Pi 4. A Raspberry Pi 4 has many up sides.
- They run on 16 watts or less.
- They can be attached to a USB connected SSD.
- They have up to 8 GB of RAM.
- They are capable of high speed internet.
- They can be automated with CRON.
The down side is mostly that they really only have just so much computing power. So the amount of currencies that can be tracked at any time is limited depending on the frequency you want to collect prices. If you want to collect data every five minutes, you could collect 5 sets of 10 currencies for a total of 50. 10 currencies will take about 50 seconds to collect and load into a database.
16 Watts
I wanted something I could power with a cheap solar panel if I wanted to. With all these high powered computers trying to chomp through data, I wanted to see if I could exploit margin trading with such low power that it would embarrass the power hungry crypto miners.
SSD
You have to use an SSD. The reason is that SD cards are not made to be written and read as often as a Raspberry Pi will read and write. I used a Western Digital Green SSD drive in a USB type C case.
As you can see above, even after 3 years of continued use, the SSD far exceeds the target performance that an SD card rarely equates to.
8 GB of RAM
It isn’t a lot. I worked for local government and used less to do their business intelligence. Older operating systems used less RAM. This changed as RasPi OS came out, but it was enough.
High Speed Internet
Not really needed. To be honest, the I didn’t need over powering amounts of internet, but it was cool that I had an onboard wi-fi adaptor.
CRON
This is probably the most important part. I have tried so many scheduling or timing libraries. They are pointless. They all work cascading rather than being able to set the program to run every minute or five minutes.
Cascading means that, if you set the program to run every minute, the schedule will run every minute plus the run time of the code. If the code runs for 46 seconds, it will take 1 minute and 46 seconds for the program to run a second time. Then, each run of the program will throw the schedule off by 46 seconds.
Cron allows you to set the time increments so that, in the case above, the program will run for 46 seconds, wait for 14 seconds, and run again, making it run every minute.
The code running every minute means that we have a set X value. For this many X minutes, the currency changed price either up or down by this Y value. Now we have a rise over run and can calculate y/x (or rise/run) to find our slope value. Now we can make predictions with some clever code and SKLearn.
Database
I tried many different storage options. I tried writing to a text file, to a CSV file, to a JSON file. Nothing performed as well as a database. Connecting to a database wasn’t the hardest thing by far. I had a harder time installing and connecting to MySQL Workbench. One thing to remember is that you have to remember to commit when inserting values into a table.
I love that I can create a function for a line of SQL in Python and run various values into or out of a MySQL database with so much ease. The programs would run for nearly a minute when I was using the Raspberry Pi for everything. I have since begun to run the code on a Linux server with an old I7-4770K processor and 16 GB of RAM. Code tends to have little limits and runs in roughly 11 seconds.
I am constantly upgrading the functionality of my platform. I track transactions, certain program history, records, predictions, and errors in the database.
Prediction
Predictions happen before midnight and are held in the database. The reason why predictions are not made more often is because the predictions are not minute to minute dependent and if they were the target buying or selling value would move with the actual value. This would likely skew the buying or selling value and miss more opportunities than it would offer. In essence, it is hard to hit a moving target.
I understand that I could perform more predictive tests on previous data based on different measures such as on a month basis that would allow for finding any correlation for certain parts of the year. I haven’t begun that testing. I do plan to in the future. Most predictive tests are based on data that is done being collected. Crypto currencies are ongoing. Therefore, I could perform more tests on a monthly basis as the data for a previous month would be done being collected.
Prediction is limited to 7 days. The predictions are updated daily. Accuracy tends to become less reliable the further its scope extends. With a daily update and historical recording, I am able to track how often my predictions are correct.
Data Collection
It didn’t take me long to realize that calling an API for data collection was not viable in so many ways.
- You can only call an API so many times. If I could pull once every minute for the entire list of traded currencies, I would have to pull it 1,440 times per day. If my limit is 800 calls per day, I will not be able to cover the entire day.
- Brokers do not always seem to act ethically. Keeping your own records is important.
So how do we handle the problem? We take the information that is publicly reported on the websites. We web scrape using BeautifulSoup.
The web scraping code is incredibly useful as a module to build the later used automated buying module to set the buying price.
Reporting
I have years of experience using Microsoft Power BI for making data report dashboards. I worked for Waukesha County as a Business Intelligence Intern. What I really was the Business Intelligence Developer and subject matter expert in charge of managing my own projects with various department heads across multiple departments, towns, and cities across the entire county as an employee limited to 20 hours a week during the school year.
We didn’t want to buy accounts for every person who would be using Power BI. We had a good server and we had a great database administration team, so we were able to try the Power BI Server, which was more experimental at the time. This gave us the ability to run our dashboards through our own intranet. I knew Power BI Server wasn’t going to be a free option for long so I looked for other options.
That brings me to today. There is a free option. It just takes work and intelligence. Of course, we work with computers so the hard work becomes a program that took me awhile to write and still needs work, admittedly, but I used python to write an html webpage and used a function to find the data use Charts.JS to build interactive graphs. I can use CRON to automate the programs and update it every minute.
The webpage created offers a summary of the currently held crypto currencies and provides 7 day predictions for each tracked currency. The limits of the program have not been tested beyond ~15 currencies so far.