Stand Apart – but be Cohesive (part 2)

Stand Apart – but be Cohesive (part 2)

Some more thoughts about standing apart, but being Cohesive.

As Cohesive Networks grows, we do struggle to keep the elements of our company that are fundamental, without being too much of a constraint on evolution and responsiveness to changing markets.

Separate from very specific policies are the stories, mantras, aphorisms that guide you. Not too long ago we had a partner company ask us to tell them why they should work with us. In response we distilled some of the essence we try to hold forth in our words and actions. Here they are with a bit of elaboration.

Our favorite code is code we never write, but we solve a customer problem.
In general people do what you pay them to do, not what you tell them to do. If your reward and identity systems are built around writing and delivering code, you get a lot of code. You get change for change’s sake. We do not do that. The expression of our brutalist approach to software development is:
– Our favorite code is code we never write, but we solve a customer problem.
– Our second favorite code is the code we removed from the most recent release.
– Our third favorite code, is that code, which sadly, exists in order to have a product.

“Run on the cloud not in the cloud.” (Avoid the economically defined architecture)
We go more in depth in this post. To boil it down, the cloud vendors have built massive feature sets that both provide value and extract payments. As a consumer of cloud one needs to ask continuously, am I getting the value?

Unfortunately, certifications in cloud expertises can become so dominant in reward and identity systems that cloud services are incorporated not to serve the ultimate customer, but to serve a certification sense of satisfaction. Customers not acutely aware of this are at risk of implementing an ‘economically defined architecture’, which is a working system architecture, but while correct, it also maximizes cloud vendor revenue. Sometimes “over the top” visible services are a better path to multi-cloud and cost control.

Networks are addresses, routes, and rules; everything else is implementation detail.
This simple statement has gotten me rapidly exited from meetings at one large scale networking vendor interested in acquiring Cohesive Networks. The product management team I was talking to took insult that I could be so reductive. That said, we as a team stand by it. There are names and numbers for things, paths to these things, and the ability to say ‘yes you can’ or ‘no you can’t’ get to those things. The question then becomes how to manifest that in a way that gives enough power to someone with in-depth network expertise, but also empowers a person “just trying to connect these two things”.


Do your best to not surface implementation detail in customer experience.

This is the corollary to above. We definitely have not succeeded at this in all parts of the VNS3 Network Platform, but we try.  Obviously we provide a networking system, so there are elements of the implementation that have to surface technical detail.

For example, there are such things as “IPsec connections” and IPsec has elaborate configuration constraints which need to be managed. That said, it can probably still be simplified. As an example, our single page UI for defining an ipsec endpoint tries to achieve this objective. I had one user tell me, “The first time I saw your IPsec UI, I wondered what college intern wrote it. After I used it for a while I decided it was genius.”

Support is the critical sales function.
Some companies will claim this to be true, and then they have the support staff hound you with upsell offers, surveys, and other actions that have NOTHING to do with solving your problems. What we mean at Cohesive is, when a customer has a question or concern, your ability to quickly and knowledgeably assist them is a critical part of your trust relationship.

There can be no ulterior motive other than you want to help them in that distinct moment. In our case where the majority of our customers are SaaS/BPaaS providers, initial connectivity issues are a key obstacle to time-to-value for both the SaaS provider and their customer. Eliminating any issues in this dimension helps our customer and their customer alike.

FNA – full network accountability is a company culture.
Multi-party networking and security by definition is a collaborative endeavor. When there is a problem, there is a risk that collaboration devolves to finger pointing. When you have a solution like the VNS3 Network Platform that is SO reliable it can be difficult to look at your self as the culprit when a customer has issues. Well over 90% of all inbound “P1” support issues that we receive come as a result of our customer’s customer has <changed misconfigured broken> something in the network path.

It is tempting to enter the interaction with a bias towards a “it’s not us, it’s you” mindset. To counter this we practice what we call “Full Network Accountability” which means to the best of our ability we will work with our customer, their customer, their customer’s outsourced networking people, whomever, to solve the problem. In doing so, our team has to start with the premise, that no matter how unlikely, our customer has run into a never-before experienced, one time only, horrible bug in Cohesive VNS3, until proven otherwise. Then we move forward through the entire connection path with customers (as desired) to find the ultimate issue.

This list is not exhaustive, but is indicative of how we have approached the long term health of the VNS3 platform and our company Cohesive Networks. Software, security and network skills are the table stakes for serving customers in our space, but how to have those skills and deliver them to customers through time and across hype cycles is the additional critical capability that we strive for.

To agent or not agent, the CrowdStrike question

To agent or not agent, the CrowdStrike question

Some thoughts on the CrowdStrike events at a broad level. We don’t provide the type of capabilities they do, so I don’t feel comfortable inveighing on what happened and how it happened.  But, we do have some thoughts on when to take the risk of deploying agents into your systems.

At Cohesive we have a standard response to customers wanting to install <fill in the blank agent> into our VNS3 Network and Security platform virtual appliances. It is always a difficult situation as it is being pushed by the security team and CISO, at what tends to be our larger, more revenue significant customers. It is difficult because our answer is a polite, yet vehement “no”.

The VNS3 control plane should only be open to an allow-listed operations ingress address. The data plane should be open to the ports and services you use our platform for. (We are often a cloud edge firewall, people vpn, data center vpn, waf, and more.) The biggest liability to the security and operational integrity of our devices would be to add an agent that can be exploited or can itself cause an outage. This has happened in the industry (but not to us) numerous times over the recent years via things like SolarWinds, Microsoft OMS agent, and now CrowdStrike to name a few.

One of the oft-used phrases at Cohesive is “horses for courses” and we use that British idiom as part of our approach to making architectural and deployment security recommendations. With respect to the panoply of agents that vendors and security teams alike argue to be installed in all your systems, we think it is quite apropos. We think you need to look at what the purpose and placement of the computing devices is in order to make this decision.

Seinfeld - His father was a mudder, his mother was a mudder

If it is a computer used by a person, especially hybrid use or remote use, how are are you going to protect it? For the most part you can’t protect the network it is on, since it is running on networks you don’t control.

There are some options outside of agenting, for example if using Windows operating system you could use group policy to force the machine onto a virtual network with no ability to access the Internet other than through your VPN’s edge, but it is bad user experience and expensive to have acceptable points of presence.

And even if forcing users onto a VPN, what about email phishing links and attachments, what about Microsoft Office macro exploits that leak into your shared drives? Super tough, and I am glad we aren’t in that business.

People make mistakes. Even well-trained, diligent people can make mistakes in where they browse to or what they click on, and you need to be able to address possible exploits there at the point of attack. So I can’t really blame someone for wanting to believe these types of products are the answer.

At Cohesive which admittedly is not a huge company, our answer has evolved from ‘Microsoft Office is disallowed’, to now ‘Windows is not allowed’. Right there the “people attack surface” is minimized. However, that is a long migration or a “never migration” for many companies. In these cases, I think “agent away” and be judicious in the vendor you choose.

What about business servers, the computers that are not the personal device of an end-user, but rather are there to perform a function like database or API serving or message queuing? Again, for Cohesive, easier in that Windows is not allowed, business services are built on Linux. That said, whether Windows or Linux, in these scenarios why are you ‘protecting’ the computer instead of the network? Those business servers are not going to click on links, browse bad websites, or download malware. They are automatons performing specific, repetitive operations.

This is where we are skeptical of the agent approach and will not allow it for our network appliances, and as applicable would not allow for business services. (CrowdStrike apparently created a similar issue previously on some Linux distributions: https://www.theregister.com/2024/07/21/crowdstrike_linux_crashes_restoration_tools)

In these cases the VNS3 Network Platform (or similar vendor offerings) is there to protect and monitor ingress and egress to the network, along with other elements of what should be thought of as a security lattice. Using cloud or data center virtualization, you have your cloud network security groups and network ACLs, along with your OS firewall, both of which should be working in tandem with network appliances like VNS3. You can then run WAF or IPS at these ingress/egress points as well – run them inside VNS3 or run them via an appropriate cloud service offering.

We think these are the rules to follow:

  • Computer primarily used by human and it has to be Windows, agent it and anti-virus it.
  • Computer primarily used by human and is Mac or Linux, anti-virus it.
  • Computer used by computers, both micro-segment and protect the network, use a security lattice. The risk from any agents are too high.
  • (OPTIONAL) Computers used by computers are ideally Linux servers using the distributions’ minimal install with at least some elements of CIS hardening.

But then you loudly proclaim, “we need visibility into the VNS3 (or other) appliances for our <DataDog, Syslog Server, SumoLogic, AlertLogic, etc..>!”

OK – you got us. We DO allow visibility to logs, network configuration and statistics, netflow, and some API calls for system status to plugins that run inside VNS3 – but don’t have full host access.

When you are running VNS3 controllers, whether a single one or a larger cluster, you are running a mini “docker cloud” inside your virtual network. We have a set of pre-built plugins as well as a published set of elements needed to make any open source, proprietary source, or commercial source that can be containerized, into a compliant plugin. This gives you the ability to put analysis and monitoring INTO the network, yet not be able to disrupt the network security device; observability with security.

(And now the sound of Cohesive patting itself on the back for this balanced approach to agenting.)

Cisco owes me a sandwich!

Cisco owes me a sandwich!

In 2010 we were so excited to meet with a team at Cisco to discuss our network function virtualization offering that had been live in the clouds since 2008.

IPv6 VNET/VPC - Grains of Sand

The folks at Futuriom (@rayno, aka R. Scott Raynovich) have put up a nice piece about Cisco’s struggles to integrate its many acquisitions and fundamentally retain relevance.  Now of course Cisco is a behemoth compared to Cohesive Networks, we are a lean and cute marsupial next to it the hulking mammal, but we do compete with them all the time and they do represent the other side of about 50% of all our customer’s connections.  (My point being we are quite familiar.)

So, way back when cloud was young – our parent company CohesiveFT developed a very strong champion at Cisco.

He loved what we did with the VNS3 platform (then the VPN-cubed appliance) and was both fascinated and skeptical that via an API you could create any over-the-top network and route traffic through it, indifferent to the actual underlying networks. You could have a flat network anywhere with dynamic routing protocols in place, a form of server motion which separated network location from network identity, thus enabling very flexible and secure application systems.  In many ways, unbeknownst even to ourselves, in 2008 we had created the first commercial cloud native NFV (network function virtualization) solution with a restful API.

We ended up invited to Cisco campus in California for what would be a pretty significant (for us) meet up with some of their key technical and business development decision makers about our virtual appliances. We were thrilled and hoped for a partnership to accelerate our business.

The calendar snippet above shows that it was a morning meeting – and maybe I have conflated events through time, as in my memory it was thereafter rescheduled into a longer, later lunch meeting (catering provided) with time for a demo and hopefully some whiteboarding with the team.

The meeting was well attended by our champion, key technical people and decision makers.

A network bridge to the compute clouds.

We launched into our pitch about how you could build networks anywhere you had access to compute and bulk network ingress/egress.  Some questions came and went – and we were thrilled to be talking about the flexibility and security that comes with network virtualization.

At about 20 minutes in, product managers were scribbling away in their notebooks, but then the “biggest” of the technical folks looked up and said “Wait, wait, this is a hardware appliance, right?”

“No, no,” we say, “it’s virtual; all software, really flexible.”

To which he said, “No one will ever put network traffic through software” and he closed his notebook, stood up, and began exiting the room.

With the exception of our champion, everyone else followed.

As we were leaving the room, there was the catering cart outside that had not yet had a chance to come in to our truncated meeting. My memory is sandwiches – but maybe it was breakfast items, regardless it was now NOT on offer, and we were exited from the building.

So, two takeaways, network virtualization was real, driven by software and took Cisco a long time to get their heads around, and I think Cisco owes me a sandwich (or perhaps a croissant).