Machine learning work is full of loops: form a hypothesis, build a pipeline, run it, read the metrics, adjust, repeat. Tangle is already where Shopify’s ML experiments run, giving engineers a shared platform to build and execute pipelines.

Tangent is an autonomous agent that orchestrates ML experimentation workflows on top of Tangle, deciding what to run, running it, and reporting back. Around Tangent we built a Linux-based platform for hosting these agents securely so they can persist, work remotely, and reach the services they need without ever holding a credential. This is done using containerized isolation, a certificate-issuing proxy, and per-instance persistent storage (details below).

What is Tangle?

Tangle is an open source, platform-agnostic ML experimentation platform with a powerful drag-and-drop visual editor. Users drag components onto a canvas, wire outputs to inputs to form a pipeline graph, and submit it for execution locally or in the cloud. A caching layer skips or reuses previously executed steps, including steps still in flight, so iteration is fast and cheap. All pipeline runs are stored forever (including graphs, components, and logs), making pipelines reproducible even after years have passed.

Because runs and caches are shared, teammates can inspect, copy, and modify each other’s pipelines in seconds, with no cloning a private notebook and hoping the environment matches. Tangle has similarities with other OSS tools like Airflow and Kubeflow Pipelines; its caching model and visual editor are what make fast experimentation loops possible. Any containerized CLI program written in any language can be used as a Tangle component, and those components exchange data as files in any format (CSV, Parquet, JSON, etc.).

Tangle composes components the way shell scripts, makefiles, and *nix pipes do. More information about Tangle is available at tangleml.com and you can immediately try it out.

What is Tangent?

Tangent is an autonomous ML engineering agent designed to accelerate your Tangle experimentation workflows. It follows the pattern Andrej Karpathy recently popularized as autoresearch. Tangent takes that idea from a single training script on one GPU to full experiment pipelines running on Tangle, with a fleet of specialized subagents, gated checkpoints, and persistent memory. You point it at a scenario, a model, a metric to improve, a dataset, and it iterates: it analyzes results, forms hypotheses, modifies pipelines, submits runs, monitors them, and synthesizes what it learned. How much you delegate is up to you. Tangent works interactively, so you can hand it a single step (build this component, debug this failed run, analyze these metrics), or you can hand off the entire loop with one command and let it run:

tangent auto

Under the hood is an eight-step loop with gated checkpoints. The agent won’t advance until every item on a step’s checklist passes, and it reloads its instructions and context at each gate so it doesn’t drift over a long run. Memory is persistent and plain-text: a MEMORY.md holds the best-known configuration and hard-won lessons; daily session logs capture the play-by-play, and per-run learnings are archived to object storage.

The Tangent Skill

Tangent’s brain is a skill, written in Markdown. The entry point is a single SKILL.md, backed by a fleet of subagent skills: a researcher, a builder, a debugger, a reviewer, and more. Because skills are just files, they’re portable, reviewable in a pull request, and harness-agnostic, the same skill drives whichever coding agent you prefer.

That portability matters because Tangent runs on open agent harnesses instead of one proprietary client. To add a capability, you write a Markdown file and commit it. There is no binary to build or ship. Markdown was chosen deliberately over a config format or DSL: it’s diffable in a normal PR, requires no schema or parser to validate, and both humans and the coding agent read it natively. The skill file doubles as its own documentation.

# Tangent
A skill is just Markdown. This file is the entry point; each subagent is
its own Markdown file, loaded on demand.

## Commands
tangent <subagent>   Read agents/<subagent>.md and follow it.
tangent auto          Run the autonomous 8-step experiment loop.

## Subagents                Agent file
tangent builder             agents/builder.md
tangent debugger            agents/debugger.md
tangent researcher          agents/researcher.md
tangent reviewer             agents/reviewer.md
tangent reporter            agents/reporter.md

## Auto Mode - the loop
0 Initialize -> 1 Analyze -> 2 Hypothesize -> 3 Submit ->
4 Monitor -> 5 Evaluate -> 6 Synthesize -> 7 Decide -> (loop)

Each step has a gate. The agent won't advance until every item on the
step's checklist passes - and it re-reads its instructions at each
gate so it doesn't drift over a long run.

Tangent Agent Hosting Platform

Our Tangent Agent Hosting Platform helps users deploy persistent Tangent instances that communicate with Tangle, cloud providers, and other external services. Each Tangent instance is a multi-agent space: a Linux-based VM/container that can host multiple agentic apps (TUI, API, WebUI). Because every Tangent component runs as a standard Linux process inside a container, Tangent inherits mature Linux networking, storage, and isolation primitives instead of introducing a custom runtime. Instance data (like agent sessions and memories) is persisted across restarts. There are also cross-instance shared memories.

Auth Proxy

Agents need access to services, but there is always a risk of agents reading the credentials and leaking them to AI providers. Tangent solves this by adding a system-wide proxy which lives in a separate container. The proxy intercepts and modifies HTTP requests coming from the agentic tools. Auth proxy automatically adds auth headers and can modify request URLs (e.g. redirect api.aicompany.com to some AI proxy). To modify HTTPS requests, auth proxy creates new SSL certificates on the fly. The agent container’s OS and programs are configured to trust those certificates via a generated certificate authority.

Implementation details

In the Kubernetes version, each instance is a StatefulSet: a container Pod running agents, apps, and proxies, backed by a per-instance PersistentVolume and a shared memory volume mount.

Tangent Shell

To work natively with Tangle, we built a custom Agent Host image. The Tangent Shell is an environment where agents run remotely, keep their memory and sessions across restarts, and keep working long after you close your laptop. Tangent Shell orchestrates multiple agents: it splits a request into slices, delegates them to parallel sub-agents, and coordinates results through a Prime agent that owns the session.

Tangent Shell is built to meet the ML expert’s needs. Each session boots pre-loaded with the Tangent skill toolkit and Tangle API tools, instructed to assist with Tangle-based ML experiments: building and optimizing ML training pipelines, testing hypotheses, ablation studies, hyperparameter optimization, etc. With the help of triggers (like webhooks, timers, and schedules), Tangent Shell is able to monitor pipeline executions and implement deep experimentation plans. The agent knows how to operate inside the rich UI, and how to render visual artifacts (Markdown, PNG, HTML).

The Shell is open source. Agent Bundles (packaged sets of prompts, tools, skills, workflows, and triggers) extend it without touching core code.

A real use case

We used Tangent to rebuild a large reranking model end to end. An engineer set the direction (which features to try, what training data to add) and reviewed results at each step. Tangent built, ran, and analyzed the experiments. Working through the loop, it tried a sequence of changes and measured each one against the previous best:

Open source and contributing

Tangle, the Tangent skills, the Hosting Platform, and Tangent Shell are all released under Apache 2.0. Development happens in the open on GitHub, and the projects accept pull requests for new subagent skills, Agent Bundles, and core fixes. Tangle is maintained by its creator, Alexey Volkov, with Shopify as the project’s initial sponsor and infrastructure steward. If you build a subagent skill or Agent Bundle you think others would find useful, we welcome the PR.

Where to find Tangle and Tangent

• Tangle website: https://tangleml.com/
• Tangle source repo: https://github.com/TangleML/tangle
• Tangle CLI and Tangent skills: https://github.com/TangleML/tangle-cli
• Tangent Agent Hosting Platform: https://github.com/TangleML/tangent
• Tangent Shell: https://github.com/TangleML/tangent-shell

The post Building Autonomous ML Experimentation with Tangle and Tangent appeared first on Linux.com.

Next week, OPF will be exhibiting at the SC22 conference in Dallas, TX.

OPF will be showcasing our LibreBMC project and talking about all things open hardware and software. The best part is that we won’t be alone, as we have a number of OPF community members and partners showcasing at our booth as well, including Open Compute Project, Axiado, Raptor Computing, Oregon State University, and TORmem.

Our friends at Open Compute Project, will be sharing OPF’s booth to showcase the breadth of open hardware solutions including the DC-SCM project, which decouples controllers and hardware resources to enable modular server management on a common form factor. OPF has adopted OCP’s DC-SCM specification for use within OPF’s own LibreBMC SIG, and has been participating within OCP’s workgroup to drive the technical roadmap and adoption of the standard. We look forward to continued collaboration between our communities, especially through hardware/software co-design initiatives and workgroups.

Axiado, an AI-enhanced hardware security company based in San Jose, CA, and the newest Silver member to the OpenPOWER Foundation, will be sharing our booth to showcase their latest security solution.

Axiado’s will be demoing their datacenter-ready secure control module (DC-SCM) Smart SCM solution, which was inspired by OpenPOWER’s very own LibreBMC project!

We recently shared OPF’s booth at the OCP Global Summit and are excited to once again share our booth with Axiado at SC22!

Another OPF Silver member, Raptor Computing, will be showcasing at our booth. Raptor is a US-based ODM for secure, owner-controlled computing solutions. They are involved in a number of initiatives here at the Foundation and will be showcasing their own Kestrel OpenPOWER-based soft BMC solution.

OPF has also recently stood up the OPF HUB SIG, which the very talented team at Oregon State University open source lab is leading.

The OSU open source lab is a nonprofit organization working for the advancement of open source technologies, and provides hosting for more than 160 projects, including those of worldwide leaders like the Apache Software Foundation, the Linux Foundation and Drupal.

The OPF HUB provides POWER based hardware and software resources for the purpose of porting, converting, and maintaining open source resources to be enabled on OpenPOWER solutions.

OSU will be at the OPF booth demoing the HUB and how one can request and utilize their openPOWER resources.

Finally, we will be sharing our booth with TORmem, a new startup focused on bringing innovative disaggregated memory technologies to market that will enable new applications, new computing system designs, and vastly improved cost efficiency and performance for customers from enterprise to hyperscalers.

TORmem’s solution utilizes the latest in open memory solutions including CXL and OpenCAPI’s Open Memory Interface solution.

We have a full house of innovative member companies and partners at our SC22 booth #3139 this year, so if you are heading to the event, please stop by and say hello!

The post OpenPOWER Foundation and Community Members to showcase at SC22 Conference in Dallas, TX! appeared first on Linux.com.

SAN FRANCISCO, March 23, 2020 — The Linux Foundation, the nonprofit organization enabling mass innovation through open source, along with co-host the TODO Group, an open group of organizations who collaborate on practices, tools and other ways to run successful and effective open source programs and projects, has opened its Call for Proposals for OSPOCon.  The event will take place September 29 – October 1 in Dublin, Ireland, alongside Open Source Summit + Embedded Linux Conference 2021. The TODO Group has also launched an OSPO Landscape as a resource for the community to learn more about OSPOs. The community is encouraged to contribute to the landscape.

OSPOCon is a new event, dedicated to those working to create a center of competency for open source in their organizations in order to join together to overcome challenges through sharing experiences, best practices, and tooling. Open Source Program Offices (OSPOs) face many obstacles, such as ensuring high-quality and frequent releases, engaging with developer communities, and contributing back to other projects effectively. Collaborating together with others working on the same concerns helps the entire ecosystem improve.

“I am thrilled to be a part of the inaugural OSPOCon and see it brought to life to support the many hardworking and dedicated people involved in creating and sustaining OSPOs,” said Chris Aniszczyk, co-founder of the TODO Group and CTO at The Linux Foundation. “The impact OSPOs are having grows every day as they become a strategic function for organizations, from companies to governments to research institutions. Their contributions are tremendously valued and we look forward to furthering their abilities to collaborate, grow, and learn from each other.”

Proposals to speak at OSPOCon are being accepted now through June 13 at 11:59pm PDT.

Submission types requested include:

• Session Presentation (~40-50 minutes in length)
• Panel Discussion (~40-50 minutes in length)
• Birds of a Feather Session (BoFs are typically held in the evenings, (~45 minutes – 1 hour in length)
• Tutorial (~1.5 hours in length)
• Lightning Talk (~5-10 minutes in length)

Suggested Topics include:

Open Source Program Management

• Creation and Best Practices of Open Source Program Offices (OSPOs)
• Consuming and Contributing to Open Source
• Managing Competing Corporate Interests while Driving Coherent Communities
• How to Vet the Viability of OS Projects
• Internal vs External Developer Adoption
• Handling License Obligations in Organizations
• Open Source Corporate Sustainability

All interested parties are welcome to submit proposals. Those submitting will be notified of a decision by Thursday, July 22. To learn more and/or submit, please click here.

OSPOCon will be presented as a hybrid event – attendees can join and participate in person or virtually. Registration will open in late Spring 2021.  To receive an email alert when registration opens, please click here. The Linux Foundation provides diversity and need-based registration scholarships for this event to anyone that needs it; for information on eligibility please click here. Visit our website and follow us on Twitter, Facebook, and LinkedIn for all the latest event updates and announcements.

Sponsor:
OSPOCon offers two sponsorship levels for your consideration, Co-host and Supporter.  To see all sponsorship benefits, please click here or email us here.

Members of the press who would like to request a media pass should contact Kristin O’Connell.

About The Linux Foundation
The Linux Foundation is the organization of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and industry adoption. Together with the worldwide open source community, it is solving the hardest technology problems by creating the largest shared technology investment in history. Founded in 2000, The Linux Foundation today provides tools, training and events to scale any open source project, which together deliver an economic impact not achievable by any one company. More information can be found at www.linuxfoundation.org.

The Linux Foundation Events are where the world’s leading technologists meet, collaborate, learn and network in order to advance innovations that support the world’s largest shared technologies.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage.

Linux is a registered trademark of Linus Torvalds.

####

Media Contact:
Kristin O’Connell
The Linux Foundation
koconnell@linuxfoundation.org

The post The Linux Foundation and the TODO Group Announce Call for Proposals for OSPOCon and the OSPO Landscape appeared first on Linux Foundation.

The post The Linux Foundation and the TODO Group Announce Call for Proposals for OSPOCon and the OSPO Landscape appeared first on Linux.com.

Solus is a fascinating Linux distribution. It’s built from scratch, falls under the category of rolling release and by default ships with the Budgie desktop environment — which was also developed by the Solus Project. Other desktop environment ISOs like Gnome and MATE are available. … Read more at Forbes

The post Solus 4 Linux Gaming Report: A Great Nvidia, Radeon And Steam User Experience appeared first on Linux.com.

Load balancing via scalable task stealing

The Linux task scheduler balances load across a system by pushing waking tasks to idle CPUs, and by pulling tasks from busy CPUs when a CPU becomes idle. Efficient scaling is a challenge on both the push and pull sides on large systems. For pulls, the scheduler searches all CPUs in successively larger domains until an overloaded CPU is found, and pulls a task from the busiest group. This is very expensive, costing 10’s to 100’s of microseconds on large systems, so search time is limited by the average idle time, and some domains are not searched. Balance is not always achieved, and idle CPUs go unused.

I have implemented an alternate mechanism that is invoked after the existing search in idle_balance() limits itself and finds nothing. I maintain a bitmap of overloaded CPUs, where a CPU sets its bit when its runnable CFS task count exceeds 1. The bitmap is sparse, with a limited number of significant bits per cacheline. This reduces cache contention when many threads concurrently set, clear, and visit elements. There is a bitmap per last-level cache. When a CPU becomes idle, it searches the bitmap to find the first overloaded CPU with a migratable task, and steals it. This simple stealing yields a higher CPU utilization than idle_balance() alone, because the search is cheap, costing 1 to 2 microseconds, so it may be called every time the CPU is about to go idle. Stealing does not offload the globally busiest queue, but it is much better than running nothing at all.

Results

Stealing improves utilization with only a modest CPU overhead in scheduler code. In the following experiment, hackbench is run with varying numbers of groups (40 tasks per group), and the delta in /proc/schedstat is shown for each run, averaged per CPU, augmented with these non-standard stats:

• %find – percent of time spent in old and new functions that search for idle CPUs and tasks to steal and set the overloaded CPUs bitmap.
• steal – number of times a task is stolen from another CPU. Elapsed time improves by 8 to 36%, costing at most 0.4% more find time.

​​CPU busy utilization is close to 100% for the new kernel, as shown by the green curve in the following graph, versus the orange curve for the baseline kernel:

Stealing improves Oracle database OLTP performance by up to 9% depending on load, and we have seen some nice improvements for mysql, pgsql, gcc, java, and networking. In general, stealing is most helpful for workloads with a high context switch rate.

The code

As of this writing, this work is not yet upstream, but the latest patch series is at https://lkml.org/lkml/2018/12/6/1253. If your kernel is built with CONFIG_SCHED_DEBUG=y, you can verify that it contains the stealing optimization using

  # grep -q STEAL /sys/kernel/debug/sched_features && echo Yes
  Yes

If you try it, note that stealing is disabled for systems with more than 2 NUMA nodes, because hackbench regresses on such systems, as I explain in https://lkml.org/lkml/2018/12/6/1250 .However, I suspect this effect is specific to hackbench and that stealing will help other workloads on many-node systems. To try it, reboot with kernel parameter sched_steal_node_limit = 8 (or larger).

Future work

After the basic stealing algorithm is pushed upstream, I am considering the following enhancements:

• If stealing within the last-level cache does not find a candidate, steal across LLC’s and NUMA nodes.
• Maintain a sparse bitmap to identify stealing candidates in the RT scheduling class. Currently pull_rt_task() searches all run queues.
• Remove the core and socket levels from idle_balance(), as stealing handles those levels. Remove idle_balance() entirely when stealing across LLC is supported.
• Maintain a bitmap to identify idle cores and idle CPUs, for push balancing.

This article originally appeared at Oracle Developers Blog.

The post Can Better Task Stealing Make Linux Faster? appeared first on Linux.com.

The post Linux Desktop News: Zorin OS 15 Gets New Touch Interface, Android Sync And Native Flatpak Support appeared first on Linux.com.

Oracle Linux kernel developer Alan Maguire presents this six-part series on BPF, wherein he presents an in depth look at the kernel’s “Berkeley Packet Filter” — a useful and extensible kernel function for much more than packet filtering.

If you follow Linux kernel development discussions and blog posts, you’ve probably heard BPF mentioned a lot lately. It’s being used for high-performance load-balancing, DDoS mitigation and firewalling, safe instrumentation of kernel and user-space code and much more! BPF does this by supporting a safe flexible programming environment in many different contexts; networking datapaths, kernel probes, perf events and more. Safety is key – in most environments, adding a kernel module introduces significant risk. BPF programs however are verified at program load time to ensure no out-of-bounds accesses occur etc. In addition BPF supports just-in-time compilation of its bytecode to the native instructions set, so BPF programs are also fast. If you’re interested in topics like fast packet processing and observability, learning BPF should definitely be on your to-do list.

Here we try to give a guide to BPF, covering a range of topics which will hopefully help developers trying to get to grips with writing BPF programs. This guide is based on Linux 4.14 (which is the kernel for Oracle Linux UEK5), so do bear that in mind as there have been a bunch of changes in BPF since, and some package names etc may differ for other distributions.

Because BPF in Linux is such a fast-moving target, I’m going to try and point you at relevant places in the kernel codebase that may help you get a sense for what the technology can do. The samples/bpf directory is a great place to look to see what others have done, but here we’ll also dig into the implementation as reference, as it may give you some ideas how to create new BPF programs. The aim here isn’t to give a deep dive into BPF internals, but rather to give a few pointers to areas in the code which reveal BPF functionality. The source tree I’m using for reference is our UEK5 release, based on Linux 4.14.35. See https://github.com/oracle/linux-uek/tree/uek5/master . Most of the functionality described can be found in any recent kernel. The bpf-next tree (where BPF kernel development happens) can be found at

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git

An important caveat; again, what is below describes the state as per the 4.14 kernel. A lot has changed since; but hopefully with the pointers into the code, you’ll be better equipped to figure out what some of these changes are!

The aim here is to be able to get to the point of working on interesting problems with BPF. However before we get there, let’s look at the various pieces and how they fit together.

The first question to ask is what can we do with BPF? What kinds of programs can we write?

To get a sense for this, let’s examine the enumerated type definition from include/uapi/linux/bpf.h https://github.com/oracle/linux-uek/blob/uek5/master/include/uapi/linux/bpf.h#L117

enum bpf_prog_type {
    BPF_PROG_TYPE_UNSPEC,
    BPF_PROG_TYPE_SOCKET_FILTER,
    BPF_PROG_TYPE_KPROBE,
    BPF_PROG_TYPE_SCHED_CLS,
    BPF_PROG_TYPE_SCHED_ACT,
    BPF_PROG_TYPE_TRACEPOINT,
    BPF_PROG_TYPE_XDP,
    BPF_PROG_TYPE_PERF_EVENT,
    BPF_PROG_TYPE_CGROUP_SKB,
    BPF_PROG_TYPE_CGROUP_SOCK,
    BPF_PROG_TYPE_LWT_IN,
    BPF_PROG_TYPE_LWT_OUT,
    BPF_PROG_TYPE_LWT_XMIT,
    BPF_PROG_TYPE_SOCK_OPS,
    BPF_PROG_TYPE_SK_SKB,
};

What are all of these program types? To understand this, we will ask the same set of questions for each program type:

• what do I do with this program type?
• how do I attach my BPF program for this program type?
• what context is provided to my program? By this we mean what argument(s) and data are provided for us to work with.
• when does the attached program get run? It’s important to understand this, as it gives a sense of where for example in the network stack a filter is applied.

We won’t worry about how you create the programs for now; that side of things is relatively uniform across the various program types.

1. socket-related program types – SOCKET_FILTER, SK_SKB, SOCK_OPS

First, let’s consider the socket-related program types which allow us to filter, redirect socket data and monitor socket events. The filtering use case relates to the origins of BPF. When observing the network we want to see only a portion of network traffic, for example all traffic from a troublesome system. Filters are used to describe the traffic we want to see, and ideally we want it to be fast, and we want to give users an open-ended set of filtering options. But we have a problem; we want to throw away unneeded data as early as possible, and to do that we need to filter in kernel context. Consider the alternative to an in-kernel solution – incurring the cost of copying packets to user-space and filtering there. That would be very expensive, especially if we only want to see a portion of the network traffic and throw away the rest.To achieve this, a safe mini-language was invented to translate high-level filters into a bytecode program that the kernel can use (termed classic BPF, cBPF). The aim of the language was to support a flexible set of filtering options while being fast and safe. Filters written in this assembly-like language could be pushed by userspace programs such as tcpdump to accomplish filtering in-kernel. See

https://www.tcpdump.org/papers/bpf-usenix93.pdf

…for the classic paper describing this work. Modern eBPF took these concepts, expanded the register and instruction set, added data structures called maps, hugely expanded the kinds of events we can attach to, and much more!

For socket filtering, the common case is to attach to a raw socket (SOCK_RAW), and in fact you’ll notice most programs that do socket filtering have a line like this:

    s = socket(AF_PACKET,SOCK_RAW,htons(ETH_P_ALL));

Creating such a socket, we specify the domain (AF_PACKET), socket type (SOCK_RAW) and protocol (all packet types). In the Linux kernel, receive of raw packets is implemented by the raw_local_deliver() function. It is called called by ip_local_deliver_finish(), just prior to calling the relevant IP protocol’s handler, which is where the packet is passed to TCP, UDP, ICMP etc. So at this point the traffic has not been associated with a specific socket; that happens later, when the IP stack figures out the mapping from packet to layer 4 protocol, and then to the relevant socket (if any). You can see the cBPF bytecodes generated by tcpdump by using the -d option. Here I want to run tcpdump on the wlp4s0 interface, filtering TCP traffic only:

# tcpdump -i wlp4s0 -d 'tcp'
(000) ldh      [12]
(001) jeq      #0x86dd          jt 2    jf 7
(002) ldb      [20]
(003) jeq      #0x6             jt 10    jf 4
(004) jeq      #0x2c            jt 5    jf 11
(005) ldb      [54]
(006) jeq      #0x6             jt 10    jf 11
(007) jeq      #0x800           jt 8    jf 11
(008) ldb      [23]
(009) jeq      #0x6             jt 10    jf 11
(010) ret      #65535
(011) ret      #0

Without much deep knowledge we can get a feel for what’s happening here. On line 000 we load the offset of the ether header + 12 ; the ether header protocol type. On line 001, we jump to 002 if it matches ETH_P_IPv6 (0x86dd) (jt 2), otherwise jump to 007 if false (jf 7) (handle the IPv4 case).

Let’s look at the IPv6 case first. On line 003 we jump to 010 – success – if the IPv6 protocol (offset + 20) is 6 (IPPROTO_TCP) – line 010 returns 65535 which is the max length so we’re accepting the packet. Otherwise we jump to 004. Here we compare to 0x2c, which indicates there’s an IPv6 fragment header. If that’s true we check if the fragment header (offset 54) specifies a next protocol value as IPPROTO_TCP, and if so we jump to 10 (success) or 11 (failure). Returning 0 means dropping the packet for filtering purposes.

Handling IPv4 is simpler; on 007 (arrived at via “jf” on 001), we check for ETH_P_IPV4 and, if found, we verify that the IPPROTO is TCP. And we’re done! Remember though this is cBPF; eBPF has an extended instruction/op set similar to x86_64 and additional registers.

One other thing to note – socket filtering is distinct from netfilter-based filtering. Netfilter defines its own set of hooks with NF_HOOK() definitions, which netfilter-based technologies such as ipfilter can use to filter traffic also. You might think – couldn’t we use eBPF there too? And you’d be right! bpfilter is replacing ipfilter in more recent Linux kernels.

So with all that in mind, let’s return to examining the socket-related program types.

1.1 BPF_PROG_TYPE_SOCKET_FILTER

• What do I do with it? The filtering actions include dropping packets (if the program returns 0) or trimming packets (if the program returns a length less than the original). See sk_filter_trim_cap() and its call to bpf_prog_run_save_cb(). Note that we’re not trimming or dropping the original packet which would still reach the intended socket intact; we’re working with a copy of the packet metadata which raw sockets can access for observability. In addition to filtering packet flow to our socket, we can also do things that have side-effects; for example collecting statistics in BPF maps.

• How do I attach my program? BPF programs can be attached to sockets via the SO_ATTACH_BPF setsockopt(), which passes in a file descriptor to the program.

• What context is provided? A pointer to the struct __sk_buff containing packet metadata/data. This structure is defined in include/linux/bpf.h, and includes key fields from the real sk_buff. The bpf verifier converts access to valid __sk_buff fields into offsets into the “real” sk_buff, see https://lwn.net/Articles/636647/ for more details.

• When does it run? Socket filters run for receive in sock_queue_rcv_skb() which is called by various protocols (TCP, UDP, ICMP, raw sockets etc) and can be used to filter inbound traffic.

To give a sense for what programs look like, here we will create a filter that trims packet data we filter on the basis of protocol type; for IPv4 TCP, let’s grab the IPv4 + TCP header only, while for UDP, we’ll take the IPv4 and UDP header only. We won’t deal with IPv4 options as it’s a simple example, so in all other cases we return 0 (drop packet).

#include <linux/bpf.h>
#include <linux/in.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include "bpf_helpers.h"

#ifndef offsetof
#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
#endif

/*
 * We are only interested in TCP/UDP headers, so drop every other protocol
 * and trim packets after the TCP/UDP header by returning length of
 * ether header + IPv4 header + TCP/UDP header.
 */

SEC("socket")
int bpf_prog1(struct __sk_buff *skb)
{
        int proto = load_byte(skb, ETH_HLEN + offsetof(struct iphdr, protocol));
        int size = ETH_HLEN + sizeof(struct iphdr);

        switch (proto) {
        case IPPROTO_TCP:
             size += sizeof(struct tcphdr);
             break;
        case IPPROTO_UDP:
             size += sizeof(struct udphdr);
             break;
        default:
             size = 0;
             break;
        }
        return size;
}

char _license[] SEC("license") = "GPL";

This program can be compiled into BPF bytecodes using LLVM/clang by specifying arch of “bpf” , and once that is done it will contain an object with an ELF section called “socket”. That is our program. The next step is to use the BPF system call to assign a file descriptor to the program, then attach it to the socket. In samples/bpf , you can see that bpf_load.c scans the ELF sections, and sections with name prefixed by “socket” are recognized as BPF_PROG_TYPE_SOCKET_FILTER programs. If you’re adding a sample I’d recommend including bpf_load.h so you can just call load_bpf_file() on your BPF program. For example, in samples/bpf/sockex1_user.c we take the filename of our program (sockex1) and load sockex1_kern.o ; the associated BPF program. Then we open a raw socket to loopback (lo) and attach the program there:

        snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
        if (load_bpf_file(filename)) {
                printf("%s", bpf_log_buf);
                return 1;
        }
        sock = open_raw_sock("lo");
        assert(setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd,
                          sizeof(prog_fd[0])) == 0);

1.2 BPF_PROG_TYPE_SOCK_OPS

• What do I do with it? Attach a BPF program to catch socket operations such as connection establishment, retransmit timeout etc. Once caught options can also be set via bpf_setsockopt(), so for example on passive establishment of a connection from a system not on the same subnet, we could lower the MTU so we won’t have to worry about intermediate routers fragmenting packets. Programs can return success (0) or failure (a negative value) and a reply value can be set to indicate the desired value for a socket option (e.g. TCP rwnd). See https://lwn.net/Articles/727189/ for full details, and look for tcp_call_bpf()s inline definition in include/net/tcp.h to see how TCP handles execution of such programs. Another use case is for sockmap updates in combination with BPF_PROG_TYPE_SK_SKB programs; the bpf_sock_ops struct pointer passed into the BPF_PROG_TYPE_SOCK_OPS program is used to update the sockmap, associating a value for that socket. Later sk_skb programs can reference those values to specify which socket to redirect to via bpf_sk_redirect_map() calls. If this sounds confusing, I’d recommend taking a look at the code in samples/sockmap.

• How do I attach my program? It is attached to a cgroup file descriptor using BPF_CGROUP_SOCK_OPS attach type.

• What context is provided? Argument provided is the context, struct bpf_sock_ops *.. Op field specifies the operatiion, BPF_SOCK_OPS_RWND_INIT, BPF_SOCK_OPS_TCP_CONNECT_CB etc. The reply field can be used to indicate to the caller a new value for a parameter set.

/* User bpf_sock_ops struct to access socket values and specify request ops
 * and their replies.
 * Some of this fields are in network (bigendian) byte order and may need
 * to be converted before use (bpf_ntohl() defined in samples/bpf/bpf_endian.h).
 * New fields can only be added at the end of this structure
 */
struct bpf_sock_ops {
    __u32 op;
    union {
        __u32 reply;
        __u32 replylong[4];
    };
    __u32 family;
    __u32 remote_ip4;    /* Stored in network byte order */
    __u32 local_ip4;    /* Stored in network byte order */
    __u32 remote_ip6[4];    /* Stored in network byte order */
    __u32 local_ip6[4];    /* Stored in network byte order */
    __u32 remote_port;    /* Stored in network byte order */
    __u32 local_port;    /* stored in host byte order */
};

• When does it run? As per the above article, unlike other BPF program types that expect to be called at a particular place in the codebase, SOCK_OPS program can be called at different places and use an “op” field to indicate that context. See include/uapi/linux/bpf.h for the enumerated BPF_SOCK_OPS_* definitions, but they include events like retransmit timeout, connection establishment etc.

1.3 BPF_PROG_TYPE_SK_SKB

• What do I do with it? Allows users to access skb and socket details such as port and IP address with a view to supporting redirect of skbs between sockets. See https://lwn.net/Articles/731133/ . This functionality is used in conjunction with a sockmap – a special-purpose BPF map that contains references to socket structures and associated values. sockmaps are used to support redirection. The program is attached and the bpf_sk_redirect_map() helper can be used to carry out the redirection. The general approach we catch socket creation events with sock_ops BPF programs, associate values with the sockmap for these, and then use data at the sk_skb instrumentation points to inform socket redirection – this is termed the verdict, and the program for this is attached to the sockmap via BPF_SK_SKB_STREAM_VERDICT. The verdict can be __SK_DROP, __SK_PASS, or __SK_REDIRECT. Another use case for this program type is in the strparser framework (https://www.kernel.org/doc/Documentation/networking/strparser.txt). BPF programs can be used to parse message streams via callbacks for read operations, verdict and read completion. TLS and KCM use stream parsing.

• How do I attach my program? A redirection progaram is attached to a sockmap as BPF_SK_SKB_STREAM_VERDICT; it should return the result of bpf_sk_redirect_map(). A strparser program is attached via BPF_SK_SKB_STREAM_PARSER and should return the length of data parsed.

• What context is provided? A pointer to the struct __sk_buff containing packet metadata/data. However more fields are accessible to the sk_skb program type. The extra set of fields available are documented in include/linux/bpf.h like so:

/* Accessed by BPF_PROG_TYPE_sk_skb types from here to ... */
__u32 family;
__u32 remote_ip4;   /* Stored in network byte order */
__u32 local_ip4;    /* Stored in network byte order */
__u32 remote_ip6[4];    /* Stored in network byte order */
__u32 local_ip6[4]; /* Stored in network byte order */
__u32 remote_port;  /* Stored in network byte order */
__u32 local_port;   /* stored in host byte order */
/* ... here. */

So from the above alone we can see we can gather information about the socket, since the above represents the key information that identifies the socket uniquely (protocol is already available in the globally-accessible portion of the struct __sk_buff).

• When does it run? A stream parser can be attached to a socket via BPF_SK_SKB_STREAM_PARSER attachment to a sockmap, and the parser runs on socket receive via smap_parse_func_strparser() in kernel/bpf/sockmap.c . BPF_SK_SKB_STREAM_VERDICT also attaches to the sockmap, and is run via smap_verdict_func().

2. tc (traffic control) subsystem programs

Next let’s examine the program type related to the TC kernel packet scheduling subsystem. See the tc(8) manpage for a general introduction, and tc-bpf(8) for BPF specifics.

2.1 tc_cls_act : qdisc classifier

• What do I do with it? tc_cls_act allows us to use BPF programs as classifiers and actions in tc, the Linux QoS subsystem. What’s even better is the tc(8) command has eBPF support also, so we can directly load BPF programs as classifiers and actions for inbound (ingress) and outbound (egress) traffic. See http://man7.org/linux/man-pages/man8/tc-bpf.8.html for a description of how to use tc’s BPF functionality. tc programs can classify, modify, redirect or drop packets.

• How do I attach my program? tc(8) can be used; see tc-bpf(8) for details. The basics are we create a “clsact” qdisc for a network device, and add ingress and egress classifiers/actoins by specifying the BPF object and relevant ELF section. Example, to add an ingress classifier to eth0 in ELF section my_elf_sec from myprog_kernel.o (a bpf-bytecode-compiled object file):

# tc qdisc add dev eth0 clsact
# tc filter add dev eth0 ingress bpf da obj myprog_kernel.o sec my_elf_sec

• What context is provided? A pointer to struct __sk_buff packet metadata/data.

• When does it get run? As mentioned above, classifier qdisc must be added, and once it is we can attach BPF programs to classify inbound and outbound traffic. Implementation-wise, act_bpf.c and cls_bpf.c implement action/classifier modules. On ingress/gress sch_handle_ingress()/egress() call tcf_classify(). In the case of ingress, we do classification via the core network interface receive function, so we are getting the packet after the driver has processed it but before IP etc see it. On egress, the filtering is done prior to submitting to the device queue for transmit.

3. xdp : the Xpress Data Path.

The key design goal for XDP is to introduce programmability in the network datapath. The aim is to provide the XDP hooks as close to the device as possible (before the OS has created sk_buff metadata) to maximize performace while supporting a common infrastructure across devices. To support XDP like this requires driver changes. For an example see drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c. A bpf net device op (ndo_bpf) is added. For bnxt it supports XDP_SETUP_PROG and XDP_QUERY_PROG actions; the former configures the device for XDP, reserving rings and setting the program as active. The latter returns the BPF program id. BPF-specific transmit and receive functions are provided and called by the real send/receive functions if needed.

3.1 BPF_PROG_TYPE_XDP

• What do I do with it? XDP allows access to packet data as early as possible, before packet metadata (struct sk_buff) has been assigned. Thus it is a useful place to do DDoS mitigation or load balancing since such activities can often avoid the expensive overhead of sk_buff allocation. XDP is all about supporting run-time programming of the kernel in via BPF hooks, but by working in concert with the kernel itself; i.e. not a kernel bypass mechanism. Actions supported include XDP_PASS (pass into network processing as usual), XDP_DROP (drop), XDP_TX (transmit) and XDP_REDIRECT. See include/uapi/linux/bpf.h for the “enum xdp_action”.

• How do I attach my program? Via netlink socket message. A netlink socket – socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE) – is created and bound, and then we send a netlink message of type NLA_F_NESTED | 43 ; this specifies XDP message. The message contains the BPF fd, the interface index (ifindex). See samples/bpf/bpf_load.c for an example.

• What context is provided? An xdp metadata pointer; struct xdp_md * . XDP metadata is deliberately lightweight; from include/uapi/linux/bpf.h:

/* user accessible metadata for XDP packet hook
 * new fields must be added to the end of this structure
 */
struct xdp_md {
        __u32 data;
        __u32 data_end;
};

• When does it get run? “Real” XDP is implemented at the driver level, and transmit/receive ring resources are set aside for XDP usage. For cases where drivers do not support XDP, there is the option of using “generic” XDP, which is implemented in net/core/dev.c. The downside of this is we do not bypass skb allocation, it just allows us to use XDP for such devices also.

4. kprobes, tracepoints and perf events

kprobes, tracepoints and perf events all provide kernel instrumentation. kprobes – https://www.kernel.org/doc/Documentation/kprobes.txt – allow instrumentation of specific functions – entry of a function can be monitored via a kprobe, along with most instructions within a function, or entry/return can be instrumented via a kretprobe. When one of these probes is enabled, the code at the enable point is saved, and replaced with a breakpoint instruction. When this breakpoint is hit a trap instruction is generated, registers are saved and we branch to the relevant instrumentation handler. For example, kprobes are handled by kprobe_dispatcher() which gets the address of the kprobe and register context as arguments. kretprobes are implemented via kprobes; a kprobe fires on entry and modifies the return address, saving the original and replacing it with the location of the instrumentation handler. Tracepoints – https://www.kernel.org/doc/Documentation/trace/tracepoints.rst – are similar, but ratther than being enabled at particular instructions, they are explicitly marked at sites in code, and if enabled can be used to collect debugging information at those sites of interest. The same tracepoint can be declared in multiple places; for example trace_drv_return_int() is called in multiple places in net/mac80211/driver-ops.c .

Perf events – https://perf.wiki.kernel.org/index.php/Main_Page – are the basis for eBPF support for these program types. BPF essentially piggy-backs on the existing infrastructure for event sampling, allowing us to attach programs to perf events of interest, which include kprobes, uprobes, tracepoints etc as well has other software events, and indeed hardware events can be monitored too.

These instrumentation points are what gives BPF the capability to be a general-purpose tracing tool as well as a means for supporting the original networking-centric use cases like socket filtering.

4.1 BPF_PROG_TYPE_KPROBE

• What do I do with it? instrument code in any kernel function (bar a few exceptions) via kprobe, or instrument entry/return via kretprobe. k[ret]probe_perf_func() executes a BPF program attached to the probe point. Note that this program type can also be used to attach to u[ret]probes – see https://www.kernel.org/doc/Documentation/trace/uprobetracer.txt for details

• How do I attach my program? When the kprobe is created via sysfs, it has an id associated with it, stored in /sys/kernel/debug/tracing/events/[uk]probe//id , /sys/kernel/debug/tracing/events/[uk]retprobe/probename/id . https://www.kernel.org/doc/Documentation/trace/kprobetrace.txt contains details on how to create a kprobe using sysfs.For example, to create a probe called “myprobe” on entry to tcp_retransmit_skb() and retrieve its id:

# echo 'p:myprobe tcp_retransmit_skb' > /sys/kernel/debug/tracing/kprobe_events
# cat /sys/kernel/debug/tracing/events/kprobes/myprobe/id
2266

We can use that probe id to open a perf event, enable it, and set the BPF program for that perf event to be our program. See samples/bpf/bpf_load.c in the load_and_attach() function for how this can be done for k[ret]probes. The code might look something like this:

        struct perf_event_attr attr;
        int eventfd, programfd;
        int probeid;

        /* Load BPF program and assign programfd to it; and get probeid of probe from sysfs */
        attr.type = PERF_TYPE_TRACEPOINT;
        attr.sample_type = PERF_SAMPLE_RAW;
        attr.sample_period = 1;
        attr.wakeup_events = 1;
        attr.config = probeid;
        eventfd = sys_perf_event_open(&attr, -1, 0, programfd, 0);
        if (eventfd < 0)
                return -errno;
        if (ioctl(eventfd, PERF_EVENT_IOC_ENABLE, 0)) {
                close(eventfd);
                return -errno;
        }
        if (ioctl(eventfd, PERF_EVENT_IOC_SET_BPF, programfd)) {
                close(eventfd);
                return -errno;
        }

• What context is provided? A struct pt_regs *ctx , from which the registers can be accessed. Much of this is platform-specific, but some general-purpose functions exist such as regs_return_value(regs), which returns the value of the register than holds the function return value (regs→ax on x86).

• When does it get run? When the probe is enabled and breakpoint is hit, k[ret]probe_perf_func() executes a BPF program attached to the probe point via trace_call_bpf(). Similar story for u[ret]probe_perf_func().

4.2 BPF_PROG_TYPE_TRACEPOINT

• What do I do with it? Instrument tracepoints in kernel code. Tracepoints can be enabled via sysfs as is the case with kprobes, and in a similar way. The list of trace events can be seen under /sys/kernel/debug/tracing/events.

• How do I attach my program? As we saw above, when the tracepoint is created via sysfs, it has an id associated with it. We can use that probe id to open a perf event, enable it, and set the BPF program for that perf event to be our program. See samples/bpf/bpf_load.c in the load_and_attach() function for how this can be done for tracepoints; the above code snippet for kprobes works for tracepoints also. As an example showing how tracepoints are enabled, here we enable the net/net_dev_xmit tracepoint as “myprobe2” and retrieve its id:

# echo 'p:myprobe2 trace:net/net_dev_xmit' > /sys/kernel/debug/tracing/kprobe_events
# cat /sys/kernel/debug/tracing/events/kprobes/myprobe2/id
2270

• What context is provided? The context provided by the specific tracepoint; arguments and data types are associated with the tracepoint definition.

• When does it get run? When the tracepoint is enabled and hit, perf_trace_() (see definition in include/trace/perf.h) calls perf_trace_run_bpf_submit() which will invoke the bpf program via trace_call_bpf().

4.3 BPF_PROG_TYPE_PERF_EVENT

• What do I do with it? Instrument software and hardware perf events. These include events like syscalls, timer expiry, sampling of hardware events, etc. Hardware events include PMU events (processor monitoring unit) which tell us things like how many instructions completed etc. Perf event monitoring can be targeted at a specific process or group, processor, and a sample period can be specified for profiling.

• How do I attach my program? A similar model as per the above; we call perf_event_open() with an attribute set, enable the perf event via PERF_EVENT_IOC_ENABLE ioctl(), and set the bpf program via PERF_EVENT_IOC_SET_BPF ioctl(). For PMU (processor monitoring unit) perf event example, see these snippets from samples/bpf/sampleip_user.c:

        ...
        struct perf_event_attr pe_sample_attr = {
                .type = PERF_TYPE_SOFTWARE,
                .freq = 1,
                .sample_period = freq,
                .config = PERF_COUNT_SW_CPU_CLOCK,
                .inherit = 1,
        };
                ...
        pmu_fd[i] = sys_perf_event_open(&pe_sample_attr, -1 /* pid */, i,
                                        -1 /* group_fd */, 0 /* flags */);
        if (pmu_fd[i] < 0) {
                fprintf(stderr, "ERROR: Initializing perf samplingn");
                return 1;
        }

        assert(ioctl(pmu_fd[i], PERF_EVENT_IOC_SET_BPF,
                     prog_fd[0]) == 0);
        assert(ioctl(pmu_fd[i], PERF_EVENT_IOC_ENABLE, 0) == 0);
        ...

• What context is provided? A struct bpf_perf_event_data *, which looks like this:

struct bpf_perf_event_data {
    struct pt_regs regs;
     __u64 sample_period;
};

• When does it get run? Depends on the perf event firing and the sample rate chosen, specified by freq and sample_period fields in the perf event attribute structure.

5. cgroups-related program types

CGroups are used to handle resource allocation, allowing or denying access to system resources such as CPU, network bandwidth etc for groups of processes. One key use case for cgroups is containers; a container’s resource access is limited via cgroups while its activities are isolated by the various classes of namespace (network namespace, process ID namespace etc). In the BPF context, we can create eBPF programs that allow or deny access. In include/linux/bpf-cgroup.h we can see definitions for execution of socket/skb programs, where __cgroup_bpf_run_filter_skb is called wrapped in a check that cgroup BPF is enabled:

#define BPF_CGROUP_RUN_PROG_INET_INGRESS(sk, skb)                  
({                                                                 
    int __ret = 0;                                                 
    if (cgroup_bpf_enabled)                                        
        __ret = __cgroup_bpf_run_filter_skb(sk, skb,               
                            BPF_CGROUP_INET_INGRESS);              
                                                                   
    __ret;                                                         
})

#define BPF_CGROUP_RUN_SK_PROG(sk, type)                       
({                                                                 
    int __ret = 0;                                                 
    if (cgroup_bpf_enabled) {                                      
        __ret = __cgroup_bpf_run_filter_sk(sk, type);              
    }                                                              
    __ret;                                                         
})

If cgroups are enabled, we attach our program to the cgroup and it will be executed at the relevant hook points. To get an idea of the full list of hooks, consult include/uapi/linux/bpf.h and examine the enumerated type “bpf_attach_type” for BPF_CGROUP_* definitions.

5.1 BPF_PROG_TYPE_CGROUP_SKB

• What do I do with it? Allow or deny network access on IP egress/ingress (BPF_CGROUP_INET_INGRESS/BPF_CGROUP_INET_EGRESS). BPF programs should return 1 to allow access. Any other value results in the function __cgroup_bpf_run_filter_skb() returning -EPERM, which will be propagated to the caller such that the packet is dropped.

• How do I attach my program? The program is attached to a specific cgroup’s file descriptor.

• What context is provided? The relevant skb.

• When does it get run? For inet ingress, sk_filter_trim_cap() (see above) contains a call to BPF_CGROUP_RUN_PROG_INET_INGRESS(sk, skb); if a non-zero value is returned, the error is propogated to the caller (e.g. __sk_receive_skb()) and the packet is discarded and freed. A similar approach is taken on egress, but in ip[6]_finish_output().

5.2 BPF_PROG_TYPE_CGROUP_SOCK

• What do I do with it? Allow or deny network access at various socket-related events (BPF_CGROUP_INET_SOCK_CREATE, BPF_CGROUP_SOCK_OPS). As above, BPF programs should return 1 to allow access. Any other value results in the function __cgroup_bpf_run_filter_sk() returning -EPERM, which will be propagated to the caller such that the packet is dropped.

• How do I attach my program? The program is attached to a specific cgroup’s file descriptor.

• What context is provided? The relevant socket (sk).

• When does it get run? At socket creation time, in inet_create() we call BPF_CGROUP_RUN_PROG_INET_SOCK() with the socket as argument, and if that function fails, the socket is released.

6. Lightweight tunnel program types.

Lightweight tunnels – https://lwn.net/Articles/650778/ – are a simple way to do tunneling by attaching encapsulation instructions to routes. The examples in the patch description make things clearer: iproute examples (without BPF):

  VXLAN:
  ip route add 40.1.1.1/32 encap vxlan id 10 dst 50.1.1.2 dev vxlan0

  MPLS:
  ip route add 10.1.1.0/30 encap mpls 200 via inet 10.1.1.1 dev swp1

So we’re telling Linux for example that for traffic to 40.1.1.1/32 addresses, we want to encapsulate with a a VXLAN ID of 10 and destination IPv4 address of 50.1.1.2. BPF programs can do the encapsulation on outbound/transmit (inbound packets are readonly). See https://lwn.net/Articles/705609/ for more details. Similarly to tc, iproute eBPF support allows us to attach the eBPF program ELF section directly:

ip route add 192.168.253.2/32 
     encap bpf out obj lwt_len_hist_kern.o section len_hist 
     dev veth0

6.1 BPF_PROG_TYPE_LWT_IN

• What do I do with it? Examine inbound packets for lightweight tunnel de-encapsulation.

• How do I attach my program? Via “ip route add”.

# ip route add <route+prefix> encap bpf in obj <bpf object file.o> section <ELF section> dev <device>

• What context is provided? Pointer to the sk_buff.

• When does it get run? Via lwtunnel_input() ; that function supports a number of encapsulation types including BPF. The BPF case runs bpf_input in net/core/lwt_bpf.c with redirection disallowed.

6.2 BPF_PROG_TYPE_LWT_OUT

• What do I do with it? Implement encapsulation for lightweight tunnels for specific destination routes on outbound. Encapsulation is forbidden

• How do I attach my program? Via “ip route add”:

# ip route add <route+prefix> encap bpf out obj <bpf object file.o> section <ELF section> dev <device>

• What context is provided? Pointer to the struct __sk_buff

• When does it get run? Via lwtunnel_output().

6.3 BPF_PROG_TYPE_LWT_XMIT

• What do I do with it? Implement encapsulation/redirection for lightweight tunnels on transmit.

• How do I attach my program? Via “ip route add”

# ip route add <route+prefix> encap bpf xmit obj <bpf object file.o> section <ELF section> dev <device>

• What context is provided? Pointer to the struct __sk_buff

• When does it get run? Via lwtunnel_xmit().

Summary

So hopefully this roundup of program types was useful. We can see that BPF’s safe in-kernel programmable environment can be used in all sorts of interesting ways! The next thing we will talk about is what BPF helper functions are available within the various program types.

Learning more about BPF

Thanks for reading this installment of our series on BPF. We hope you found it educational and useful. Questions or comments? Use the comments field below!

Stay tuned for the next installment in this series, BPF Helper Functions.

This article originally appeared at Oracle Developers Blog

The post BPF: A Tour of Program Types appeared first on Linux.com.

Perhaps the most notable upgrade is the appearance of Budgie 10.5, even before it lands on the slick desktop environment’s official Ubuntu flavor next month. I first experienced Budgie during my review of the InfinityCubefrom Tuxedo Computers, and I found a lot to love about it. … Read more at Forbes

The post Linux Desktop News: Solus 4 Released With New Budgie Goodness appeared first on Linux.com.

Despite its size, Tuxedo crams in some powerful components. … Read more at Forbes

The post Tuxedo InfinityCube v9 Linux PC Review: Small Size, Big Possibilities appeared first on Linux.com.

The post A Linux Noob Reviews: The openSUSE Leap 15.0 Installer appeared first on Linux.com.