From Qualcomm to the Solana Foundation – a big interview with Dan Albert

At the end of June, the Incrypted Conference took place in Kiev, where one of the speakers was Dan Albert, Executive Director of the Solana Foundation. In his conversation with Incrypted, he explained why he decided to join the project in 2019, what technologies are key for blockchain, and in what direction the network will develop in the future.

This is an edited version of the interview. Watch the original on our YouTube channel.

Dan, you joined Solana before the blockchain mainnet was launched. Tell us what that was like. When did you become interested in blockchain and cryptocurrencies?

I’ve been following the cryptocurrency industry since 2017. I moved to Solana full-time in 2019, shortly after the project launched. At the time, I was working at Qualcomm.

All of the co-founders of Solana, and most of the early developers, we all came from Qualcomm. Qualcomm is a big technology company, they make chips for mobile phones and mobile networks. So a lot of us came in with knowledge of distributed systems, because mobile networks are inherently distributed systems.

So Anatoly [Yakovenko, co-founder of Solana], myself, and a few other people, we all worked at Qualcomm]. And one day in 2018, Greg [Fitzgerald, co-founder of Solana] invited me to lunch and said: “Hey, I have a friend. His name is Anatoly. He’s a super smart guy. He has this idea about how blockchain is going to change the world. We’re building a company around this idea. Do you want a job?” 

Then I said no. I said no, not because I wasn’t interested, because I was actually getting ready to quit my job anyway. I was doing personal things. I had a seasonal job – a contract job with the U.S. government at a research center in Antarctica. It was something that I had wanted to do for years.

I actually worked in Antarctica for a couple of months, installing radio equipment in different places around the continent. It was an incredible adventure. At the beginning of 2019, my contract was up, and I came back to the States and started working with the Solana team. There were about 10 or 12 of us at the time.

Judging from the photo you showed at the conference, Solana was literally a “garage startup”. What did it look like? Did you have an office or was everything literally done in a garage?

We had a small office in San Francisco at the time. I lived and still live in Colorado. We had about four, maybe five people, all living in Colorado. We were working out of our founder Greg’s basement that first summer of 2019. 

That’s where we built the first validators. That’s where we first figured out, what are the real hardware requirements? What are the performance characteristics of this network? The code base was still coming together. The runtime wasn’t fully built. The whole Solana Virtual Machine (SVM), which supports all the smart contracts today, was basically a prototype. 

But processing early transactions, doing this super fast consensus that Solana is known for, those were the things that were functional at the time. And that was enough for us to figure out: “Okay, how fast can this technology really go?”. We bought a bunch of computer parts, shipped them all over to Greg’s house, and spent a couple of months just tinkering. We tried a bunch of different configurations and finally put together six or seven of these boxes that had the right performance characteristics.

We put them together in the back of my car and drove them over to a data center in the area and put them in the rack. And that was the first physical production installation of any Solana validators.

And how do you feel today when you see how big the network has become?

It’s wild. It’s incredibly humbling to go from a time when I knew the first name of every single validator on the network because we started. There were maybe 20 or 30 validators.

I would call individual people, right? Okay, I know the head of this one, this one, and this one. And the Everstake team was one of our very first validators. They’re here in Ukraine. And it’s been incredibly humbling. There is really no other way to say it. 

Solana now has over 5,000 nodes around the world, run by huge communities. I still do my best to stay connected to all the different things that are happening. But the reality is the whole project – not just the validators and the network. It is the ecosystem, the community, all the developers, all the integrations, researchers, and people who have different opinions about what we should be doing and how we should be moving the protocol forward.

There’s so much going on that it’s almost impossible for any one person to keep track of. And that means it’s really taken on a life of its own. It’s taken on several different lives of its own. And it’s a community effort. It’s a collective massive effort. And so I’m just so grateful to everyone who’s played a part and felt like it was something.

I don’t think there are many people who don’t know what Solana is, but I have a great opportunity to ask the executive director of the Solana Foundation – what is it? 

Solana is the most scalable blockchain in the world. Solana was designed to be, and continues to be, the only place for mass adoption of blockchain-based technologies. This is due to Solana’s high speed, low fees, and most importantly, a single global computer, a single shared state machine that allows for unlimited composability and interoperability between different applications. 

While also allowing isolation, if desired, between different applications, different fee markets, and different types of computation. Without having to rely on things like sharding or L2s or things that break the state and break the execution environment, because of Solana’s performance and the parallel execution engine that really underpins the runtime, the SVM. So Solana is uniquely positioned to be the most widely used and the most powerful blockchain on the planet.

As for parallel execution, how does it work? Can this architecture be considered the main feature of the project? Is it the reason for the high throughput?

It is definitely one of the main reasons. There are, I think, two main areas, and this is going to be at a very high level, Solana’s consensus mechanism is unique. It’s quite different from a lot of EVM-based chains and projects. The parallel execution engine basically allows the Solana runtime, which is actually processing your transactions and figuring out the outputs, to scale with the hardware. 

If you can run things in parallel, you can just use more cores, more CPU, and get more parallelism over time. This scaling with hardware, I think, is the killer feature of Solana, because hardware is always getting faster. Computers keep getting faster. Computers add more cores. We get more transistors per chip. We get faster Internet. 

All of these things are going to happen with or without Solana. As long as people on Earth want to use the Internet, and as long as Intel, AMD, and NVIDIA continue to compete with each other, chips will continue to get faster, and the Internet will continue to get faster. Since that will happen regardless, that means Solana will continue to get faster and continue to scale.

Solana has always been committed to high bandwidth, but in the past the network has experienced frequent outages. Recently they have become much less frequent. What were the causes of the outages? And how did you resolve the problem?

Yes, it’s no secret. The network has struggled under some loads in the past. Certainly during the NFT craze in 2021 and 2022, there were a couple of instances where the network went down. This is kind of a blessing and a curse. Downtime sucks. It sucks for everybody. It sucks for the users, it sucks for the developers, it sucks for all the engineers. No question about it.

But the reality is, no other production blockchain has seen the load of transaction demand that Solana has seen. There are times when we’ve had to learn in production that there are implementation bugs or limitations. What I think is most important to recognize is that the challenges Solana has faced are due to unprecedented levels of demand, user demand, and traction on any blockchain, and are not a reflection of architectural flaws or architectural limitations. 

Software is written by humans. Humans are imperfect. All software has bugs. That’s not an excuse. It is simply reality. The difference between fixing an implementation bug and fixing an implementation limitation in the validator code is a manageable problem. It’s a big problem, but it’s a manageable problem, which is very different from saying: “Oh, the architecture was designed incorrectly. We need to rethink the way the protocol works. 

That is not to say it was right or wrong, but that is what we saw with the merge on Ethereum. It was a major architectural change. Ethereum went from Proof-of-Work to Proof-of-Stake. It was years in the making. It was hotly contested, and rightly so. It took an incredible amount of effort and the entire Ethereum community, because it was an architectural shift in the way the blockchain works. Solana, while there are a lot of improvements and incremental things and little bugs that need to be fixed, the architecture continues to scale and we continue to increase reliability and performance along the way.

Solana Labs has a lot to say about Jump Crypto’s Firedancer. Why is a new client so important to the project?

I think Firedancer is probably the most exciting project happening on the Solana network today at the actual blockchain level. Firedancer is a brand new validator for Solana. There are two primary validators on the Solana network today. There’s the original validator that was created by Solana Labs. This team of engineers left Solana Labs. They started a new company called Anza. Their validator is now called Agave, which is a fork of the original Labs client. There’s the Anza team, which maintains their continuation of the original client. Then there’s Jeto, which is a MEV-oriented validator. Jeto is a fork of the Agave client.

These two code bases share much of the same code and some minor differences. Firedancer is new. The Jump Crypto team that is developing Firedancer is rewriting the validator from the ground up in C, which is about as powerful a low-level language as you can get.

So Firedancer is hyper-optimized for performance. The Solana architecture scales with the hardware. The Solana architecture is incredibly performant, and that is different when you look at the existing validators today. When we launched the network four years ago, we didn’t have smart contracts, we didn’t have tokens, we didn’t have NFTs. There were no stakes rewards. There are a lot of things that have been added and built as the protocol has evolved. 

As a result, there is a certain amount of technical debt that naturally accumulates because this is production code. So every time a new feature is added, it has to be done very carefully. Firedancer has the advantage right now of having zero technical debt. And so they can look at all the things that are working well, all the implementations that need to be cleaned up or reworked and rebuilt from the ground up. It looks at the lowest level of hardware. How many cores does it have? How much memory does it have? What is the physical layout of the cores, so that the movement of data on the machine takes the shortest physical path from a CPU core to a memory register to an SSD to the network card?

There’s no project in cryptography that I know of that takes this level of hardware optimization so seriously. The number of bottlenecks they’ve removed just from the implementation challenges and their hardware optimizations provides a huge overhead in the performance of this new validator running on the same type of machine that today’s validators running Anza or Jito use.

When will Firedancer go live?

I hope we’ll see an early rollout of Firedancer on the mainnet sometime later this year in 2024.

What percentage of validators do you expect to migrate to the new software, and how important is it to have a high percentage?

I think the responsible way to handle this from the validator community is to really have a phased rollout. Everyone is excited about Firedancer and that’s great. I think I’m looking forward to a successful launch from this team. This is the software that runs the entire blockchain. If there’s an unforeseen problem, the validators either need to know that they’re running a backup node with the existing Anza validator. So that they can easily fall back if they have a local issue, or if there is a broader issue that has to do with the Firedancer integration. So that we don’t have 90% of all validators on day one just saying: “Screw it, we’re Firedancer nodes now”. 

There’s some operational learning that they have to do. Validators have years of experience running Solana Labs software. Firedancer, there are different commands to use, and there are different configurations to use. The way they do logging and debugging is a little different. It would just be up to the validators to spend some time learning how to run this new machine on the testnet or on a local network. We’ll probably see a gradual phase-in of validators over a number of months.

Additionally, one of the other benefits of Firedancer, or really a second validator in general, is that this makes Solana the only other blockchain outside of Ethereum and Bitcoin to have a true second validator implementation. Multiple validators are a huge step in improving the overall resilience of the network. You have two sets of software written by two independent groups of people.

Real decentralization.

That’s real decentralization, right. It’s written in two different languages. There are two sets of dependencies, even two different compiles or toolchains in case there’s a bug and some weird little seed dependency or rust dependency. This removes any single point of failure from the whole validator development and operation stack.

Solana Labs launched token extensions in January. Tell us more about them.

The token extensions are really exciting. This is a set of smart contracts that are being deployed. They’re live in production on the Solana mainnet today. Previously, there was a token program on Solana that represented any fungible token. We call it the SPL token program. It had basic functionality. You can mint, burn, and then of course create any token you want, and you can freely transfer it to any wallet. 

If you wanted more advanced functionality from your tokens, you would have to write your own app, write your own smart contract, and build whatever logic you want for your product. With token extensions, a lot of advanced token behavior and logic comes out of the box with a single smart contract. Now when someone launches a token, if they want to use token extensions, it’s all completely optional. It’s simple, you just add a flag on the command line or an argument to your program where you create your token and say: “I want to add this token extension” or “I want to add this token extension”. Just like that, just like adding it to a configuration file, you have additional functionality.

Will this model replace the SPL standard or is it more of an extension of the standard?

It’s an extension to the standard. It is a separate program on the chain, but it is a superset of all the standard SPL token features. Everything you can do with the SPL token, you can do with the extension and more. All the same commands and workflows you can do with a traditional token, you can do with an extended token. The SPL token program and standard are not going anywhere. I expect people to use them for a very long time, and these things can happily coexist on the network.

Am I understanding correctly that token expansion could be a driver for the growth of the stablecoin sector on Solana? And how important is it that a lot of this liquidity comes online? 

It’s certainly possible. I think there are several stablecoin projects that are currently using Solana or are considering starting on Solana that want to have some control over the supply, issuance and transferability of the stablecoin. And with token extensions, the issuer of a particular token, whether it’s a stablecoin or something else, has the ability to bake in certain logic. 

Now, that could be seen as overly controlling, and so I wouldn’t want that on a fully decentralized token like Sol, for example. But for certain things, for example, for institutions that want to make a stablecoin that can be used for business-to-business settlement. From this bank to that bank or from this vendor to this supplier. 

And they just want to make sure that they don’t accidentally send tokens to the wrong place. And so with token extensions, you can set up a whitelist of addresses. You can do things like require a memo for financial audit purposes so that every transfer has a memo. With the token hooks and the token extension, you have the ability to call any smart contract, any smart contract, every time a token is transferred.

If I have a token that has a token extension and I just send you a token without connecting to an app or going to a third party website, I just send you the token and the blockchain recognizes the transfer and executes some custom logic. There are a lot of really interesting things going on. There are like 12 different token extensions. I’m not going to describe all of them here, but it allows for a lot of really complex business logic.

The next question is about MEV. How critical is it for Solana and how do you plan to combat it?

MEV is a tricky subject. I believe there are forms of healthy MEV. There are unhealthy forms of MEV. As an overall concept, I don’t think MEV is a negative or a bad thing. I think, especially in the illiquid environments that we see, especially with a lot of the meme coins and so on.

There are opportunities where if somebody is trying to buy something and they’re not getting what they’re expecting or they’re not getting what they’re asking for and what they’re being quoted, that’s problematic. Healthy arbitrage is also a form of MEV. If there is an imbalance in pricing between this DEX and that DEX, or a token pair here and a token pair there, just like in foreign exchange, someone should and does take that opportunity to rebalance and make sure that we have fair pricing. 

I think what we probably need is a more competitive MEV environment. That should ultimately keep prices the fairest and fees the lowest for most people. Nobody is in a position to dictate what happens on the network or what doesn’t happen on the network. But where we start to see challenges is where there’s a certain type of behavior that people want to do, and there’s only one way to do it. It can be exploitative or extractive, until someone comes along and finds a better way, and then they naturally level off.

How do you see the hype around meme tokens affecting the Solana ecosystem?

I think meme coins are great in general. I don’t want to see anyone in a situation where people are getting rough or pump and dump or whatever. I’m not saying yay or nay for any particular token, but meme coins represent a couple of things. I think one, it’s just people having fun. Solana is easy, it’s fast, it’s cheap. You want to play with a few dollars on Pump.Fun? That’s your choice. Go have some fun. 

What meme coins do, and we saw this with the NFT craze a couple of years ago when all these PFP NFT projects popped up, is create this fertile breeding ground for experimentation. What are the craziest things we can do with this new technology and its current level of performance? When the NFT standards were first created, people didn’t really know what to do with it. And then a few places figured out how to do NFT marketplaces and PFP drops. And all of a sudden there was this crazy boom, and then everyone was doing everything with token-gated stuff: “Oh, you have to have this collection to get here. You have to have this much volume in my meme coin to get more points to do this, to do that.

And even though a lot of these are purely speculative games at this point, it’s the place where this rampant experimentation is accepted. And even if 99% of these ideas turn out not to be great ideas, the 1% could be the ones that really push the space forward. So once some of the hype and craziness settles down, the best ideas rise to the top, and we start to see much more attractive, long-term, sustainable projects and new products.

Don’t you think memcoins seem like a protest against sophisticated technology and all that? 

Yeah, I think that’s true, that’s what people want, right? And I think that goes for all crypto. Meme coins, I think, really turn that up to 11. Where so many people, smaller investors or just regular people who are working hard and trying to get ahead in their own lives, feel so shut out of the opportunities that people in the traditional financial system have. The Wall Street of the world. 

And people are like: “I’ll try anything to get a chance because I feel like I don’t have a chance” or “I’m working so hard and there’s all these opportunities and they always go to the same people. But what about me? I think that culturally drives some of that very early speculation. 

We saw it with NFTs a few years ago. We’re seeing it now with meme coins. People want a shot. Should their shot necessarily be in a full casino? I would rather not. But maybe that’s an immature place to start, because that’s the dream of crypto. To create an accessible market for capital, for labor, for opportunity, for talent. To flow to the people who are willing to put in the effort and make the result.

What do you think about the restaking boom? Segment has had quite an impact on Ethereum, but what about Solana?

I’m neutral on the restaking for Solana. I think there are some ways in which it might be helpful. Maybe some people fall into a little bit of a thought trap. Ethereum has its design and architecture and has its tradeoffs and limitations. Solana has a different design and a different set of tradeoffs and limitations. So the two architectures are not the same. And so when people say: “Oh, restaking is on Ethereum, it’s for a particular set of”.

That in itself is not a reason to say: “Oh, now we should do restaking on Solana”. There are a number of things that Ethereum has done in their community and in their infrastructure to scale. Solana doesn’t have those kinds of scaling challenges. The Solana Foundation gives grants for research and development of the Solana protocol. We work with all the validator teams, Jido and Anza and Firedancer, and a number of others who are making proposals, writing code, and advancing the Solana protocol. Ultimately, the adoption and use of that code is up to the community. 

We just bring people together and make sure that the right people are having the right conversations and that their proposals are moving forward. Driving R&D, supporting how we can decentralize the network, and just generally helping people understand the potential value that all of these technologies can bring to the world.

You said you’ve been to Ukraine before – in 2019. What has changed since then? I mean the ecosystem, the level of Solana developers. 

I think the level of excitement and even awareness of Solana is worlds apart. When I was first in Kyiv in the summer of 2019, this was six months before the mainnet was even launched. We had a testnet that barely worked, about a dozen validators. That was actually the very first one. That was very early in my Solana trip. The event that we did in Kyiv, which was actually the very first Solana event that I ever spoke at, was right here in Kyiv. In Ukraine. 

The Solana community was people who had heard about this guy, Anatoly, and this white paper and this idea called Proof-of-History. We didn’t have NFTs and meme coins and deep in and all the other amazing things that are happening on Solana today. I went up there and gave a deep talk on how the Solana consensus algorithm works. And what PoH is at a deep technical level, because we didn’t have much of a community, we didn’t have much of a product. It was this idea that was really powerful, but it wasn’t live yet. 

Now to come back five years later, you told me, I had the opportunity, thank you, to speak yesterday morning at the Incrypted conference. There were 2,000 people that showed up, and obviously this wasn’t just a Solana conference. This was industry-wide. But the amount of, I think, awareness of Solana, everybody who came and said, “Oh, I’ve used Pump.Fun or Solana NFTs.

I think I’m just so grateful. Everyone here is so grateful for the support. The Ukrainian developer community is known around the world as being really big and really powerful. And I think it’s just great to be able to highlight that the Ukrainian community has been with Solana from the very, very beginning and that Solana is still here with Ukraine.

Solana hackathon KUMEKATHON was held within the Ukrainian Blockchain Week. Did you have a chance to look at the projects, to assess their level? 

I only had a short time. I was able to chat with a few teams. The Kumeka team, which is Solana’s community growth, super team. They organized the KUMEKATHON, I think they registered 52 projects. 

We also recently had the global online Solana Hackathon called Renaissance. There were thousands of entries from all over the world. Ukraine, I think, was in the top 10 countries in terms of submissions. So clearly there’s a lot of people building on Solana or wanting to build on Solana. So it’s really exciting to see.

It’s really great. And your visit to Ukraine means a lot to us, to the community. And not only for us – it is a signal to the whole world. That Ukraine is still here, has stood and will stand. 

Why did you decide to come to Ukraine? There is a war going on and probably 60% of the speakers will refuse.

Thank you. Yes, we had that conversation because we had… There were a number of other people from the foundation who were thinking about coming to Ukraine, but maybe for similar reasons they were not able to come. I had such fond memories of being here before. Some sort of mental connection remained.

I feel like it feels really good for me personally to be here at the very, very beginning of Solana. Now it’s such an important chapter in the history of Solana. It feels like it’s such an important chapter in the history of Ukraine that I wanted to come in person. To meet everybody and remind them that Ukraine is still here, our community is still here, and it’s super, super important.