Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,27 @@
# Changelog

## Unreleased

### Added

- **PID serialization** - Erlang PIDs now convert to `erlang.Pid` objects in Python
and back to real PIDs when returned to Erlang. Previously, PIDs fell through to
`None` (Erlang→Python) or string representation (Python→Erlang).

- **`erlang.send(pid, term)`** - Fire-and-forget message passing from Python to
Erlang processes. Uses `enif_send()` directly with no suspension or blocking.
Raises `erlang.ProcessError` if the target process is dead.

- **`erlang.ProcessError`** - New exception for dead/unreachable process errors.
Subclass of `Exception`, so it's catchable with `except Exception` or
`except erlang.ProcessError`.

### Changed

- **`SuspensionRequired` base class** - Now inherits from `BaseException` instead
of `Exception`. This prevents ASGI/WSGI middleware `except Exception` handlers
from intercepting the suspension control flow used by `erlang.call()`.

## 1.8.1 (2026-02-25)

### Fixed
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ Key features:
- **Dirty NIF execution** - Python runs on dirty schedulers, never blocking the BEAM
- **Elixir support** - Works seamlessly from Elixir via the `:py` module
- **Bidirectional calls** - Python can call back into registered Erlang/Elixir functions
- **Type conversion** - Automatic conversion between Erlang and Python types
- **Message passing** - Python can send messages directly to Erlang processes via `erlang.send()`
- **Type conversion** - Automatic conversion between Erlang and Python types (including PIDs)
- **Streaming** - Iterate over Python generators chunk-by-chunk
- **Virtual environments** - Activate venvs for dependency isolation
- **AI/ML ready** - Examples for embeddings, semantic search, RAG, and LLMs
Expand Down
139 changes: 138 additions & 1 deletion c_src/py_callback.c
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,50 @@ static PyObject *ErlangFunction_New(PyObject *name) {
return (PyObject *)self;
}

/* ============================================================================
* ErlangPid - opaque wrapper for Erlang process identifiers
*
* ErlangPidObject is defined in py_nif.h for use by py_convert.c
* ============================================================================ */

static PyObject *ErlangPid_repr(ErlangPidObject *self) {
/* Show the raw term value for debugging — not a stable external format,
but distinguishes different PIDs in logs and repls. */
return PyUnicode_FromFormat("<erlang.Pid 0x%lx>",
(unsigned long)self->pid.pid);
}

static PyObject *ErlangPid_richcompare(PyObject *a, PyObject *b, int op) {
if (!Py_IS_TYPE(b, &ErlangPidType)) {
Py_RETURN_NOTIMPLEMENTED;
}
ErlangPidObject *pa = (ErlangPidObject *)a;
ErlangPidObject *pb = (ErlangPidObject *)b;
int eq = enif_is_identical(pa->pid.pid, pb->pid.pid);
switch (op) {
case Py_EQ: return PyBool_FromLong(eq);
case Py_NE: return PyBool_FromLong(!eq);
default: Py_RETURN_NOTIMPLEMENTED;
}
}

static Py_hash_t ErlangPid_hash(ErlangPidObject *self) {
Py_hash_t h = (Py_hash_t)enif_hash(ERL_NIF_PHASH2, self->pid.pid, 0);
if (h == -1) h = -2; /* -1 is reserved for errors in Python */
return h;
}

PyTypeObject ErlangPidType = {
PyVarObject_HEAD_INIT(NULL, 0)
.tp_name = "erlang.Pid",
.tp_basicsize = sizeof(ErlangPidObject),
.tp_flags = Py_TPFLAGS_DEFAULT,
.tp_repr = (reprfunc)ErlangPid_repr,
.tp_richcompare = ErlangPid_richcompare,
.tp_hash = (hashfunc)ErlangPid_hash,
.tp_doc = "Opaque Erlang process identifier",
};

/**
* Python implementation of erlang.call(name, *args)
*
Expand Down Expand Up @@ -911,6 +955,68 @@ static PyObject *erlang_call_impl(PyObject *self, PyObject *args) {
return NULL;
}

/* ============================================================================
* erlang.send() - Fire-and-forget message passing
*
* Sends a message directly to an Erlang process mailbox via enif_send().
* No suspension, no blocking, no reply needed.
* ============================================================================ */

/**
* @brief Python: erlang.send(pid, term) -> None
*
* Fire-and-forget message send to an Erlang process.
*
* @param self Module reference (unused)
* @param args Tuple: (pid:erlang.Pid, term:any)
* @return None on success, NULL with exception on failure
*/
static PyObject *erlang_send_impl(PyObject *self, PyObject *args) {
(void)self;

if (PyTuple_Size(args) != 2) {
PyErr_SetString(PyExc_TypeError,
"erlang.send requires exactly 2 arguments: (pid, term)");
return NULL;
}

PyObject *pid_obj = PyTuple_GetItem(args, 0);
PyObject *term_obj = PyTuple_GetItem(args, 1);

/* Validate PID type */
if (!Py_IS_TYPE(pid_obj, &ErlangPidType)) {
PyErr_SetString(PyExc_TypeError, "First argument must be an erlang.Pid");
return NULL;
}

ErlangPidObject *pid = (ErlangPidObject *)pid_obj;

/* Allocate a message environment and convert the term */
ErlNifEnv *msg_env = enif_alloc_env();
if (msg_env == NULL) {
PyErr_SetString(PyExc_MemoryError, "Failed to allocate message environment");
return NULL;
}

ERL_NIF_TERM msg = py_to_term(msg_env, term_obj);

if (PyErr_Occurred()) {
enif_free_env(msg_env);
return NULL;
}

/* Fire-and-forget send */
if (!enif_send(NULL, &pid->pid, msg_env, msg)) {
enif_free_env(msg_env);
PyErr_SetString(ProcessErrorException,
"Failed to send message: process may not exist");
return NULL;
}

enif_free_env(msg_env);
Py_RETURN_NONE;
}

/* ============================================================================
* Async callback support for asyncio integration
*
Expand Down Expand Up @@ -1288,6 +1394,10 @@ static PyMethodDef ErlangModuleMethods[] = {
"Call a registered Erlang function.\n\n"
"Usage: erlang.call('func_name', arg1, arg2, ...)\n"
"Returns: The result from the Erlang function."},
{"send", erlang_send_impl, METH_VARARGS,
"Send a message to an Erlang process (fire-and-forget).\n\n"
"Usage: erlang.send(pid, term)\n"
"The pid must be an erlang.Pid object."},
{"_get_async_callback_fd", get_async_callback_fd, METH_NOARGS,
"Get the file descriptor for async callback responses.\n"
"Used internally by async_call() to register with asyncio."},
Expand Down Expand Up @@ -1344,6 +1454,11 @@ static int create_erlang_module(void) {
return -1;
}

/* Initialize ErlangPid type */
if (PyType_Ready(&ErlangPidType) < 0) {
return -1;
}

PyObject *module = PyModule_Create(&ErlangModuleDef);
if (module == NULL) {
return -1;
Expand All @@ -1353,7 +1468,7 @@ static int create_erlang_module(void) {
* This exception is raised internally when erlang.call() needs to suspend.
* It carries callback info in args: (callback_id, func_name, args_tuple) */
SuspensionRequiredException = PyErr_NewException(
"erlang.SuspensionRequired", NULL, NULL);
"erlang.SuspensionRequired", PyExc_BaseException, NULL);
if (SuspensionRequiredException == NULL) {
Py_DECREF(module);
return -1;
Expand All @@ -1365,6 +1480,20 @@ static int create_erlang_module(void) {
return -1;
}

/* Create erlang.ProcessError for dead/unreachable processes */
ProcessErrorException = PyErr_NewException(
"erlang.ProcessError", NULL, NULL);
if (ProcessErrorException == NULL) {
Py_DECREF(module);
return -1;
}
Py_INCREF(ProcessErrorException);
if (PyModule_AddObject(module, "ProcessError", ProcessErrorException) < 0) {
Py_DECREF(ProcessErrorException);
Py_DECREF(module);
return -1;
}

/* Add ErlangFunction type to module (for introspection) */
Py_INCREF(&ErlangFunctionType);
if (PyModule_AddObject(module, "Function", (PyObject *)&ErlangFunctionType) < 0) {
Expand All @@ -1373,6 +1502,14 @@ static int create_erlang_module(void) {
return -1;
}

/* Add ErlangPid type to module */
Py_INCREF(&ErlangPidType);
if (PyModule_AddObject(module, "Pid", (PyObject *)&ErlangPidType) < 0) {
Py_DECREF(&ErlangPidType);
Py_DECREF(module);
return -1;
}

/* Add __getattr__ to enable "from erlang import name" and "erlang.name()" syntax
* Module __getattr__ (PEP 562) needs to be set as an attribute on the module dict */
PyObject *getattr_func = PyCFunction_New(&getattr_method, module);
Expand Down
17 changes: 17 additions & 0 deletions c_src/py_convert.c
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,12 @@ static ERL_NIF_TERM py_to_term(ErlNifEnv *env, PyObject *obj) {
return result;
}

/* Handle ErlangPid → Erlang PID */
if (Py_IS_TYPE(obj, &ErlangPidType)) {
ErlangPidObject *pid_obj = (ErlangPidObject *)obj;
return enif_make_pid(env, &pid_obj->pid);
}

/* Handle NumPy arrays by converting to Python list first */
if (is_numpy_ndarray(obj)) {
PyObject *tolist = PyObject_CallMethod(obj, "tolist", NULL);
Expand Down Expand Up @@ -528,6 +534,17 @@ static PyObject *term_to_py(ErlNifEnv *env, ERL_NIF_TERM term) {
return dict;
}

/* Check for PID */
{
ErlNifPid pid;
if (enif_get_local_pid(env, term, &pid)) {
ErlangPidObject *obj = PyObject_New(ErlangPidObject, &ErlangPidType);
if (obj == NULL) return NULL;
obj->pid = pid;
return (PyObject *)obj;
}
}

/* Check for wrapped Python object resource */
py_object_t *wrapper;
if (enif_get_resource(env, term, PYOBJ_RESOURCE_TYPE, (void **)&wrapper)) {
Expand Down
3 changes: 3 additions & 0 deletions c_src/py_nif.c
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,9 @@ _Atomic uint64_t g_callback_id_counter = 1;
/* Custom exception for suspension */
PyObject *SuspensionRequiredException = NULL;

/* Custom exception for dead/unreachable processes */
PyObject *ProcessErrorException = NULL;

/* Cached numpy.ndarray type for fast isinstance checks (NULL if numpy not available) */
PyObject *g_numpy_ndarray_type = NULL;

Expand Down
7 changes: 7 additions & 0 deletions c_src/py_nif.h
Original file line number Diff line number Diff line change
Expand Up @@ -707,6 +707,13 @@ extern _Atomic uint64_t g_callback_id_counter;
/** @brief Python exception class for suspension */
extern PyObject *SuspensionRequiredException;

/** @brief Python exception for dead/unreachable process */
extern PyObject *ProcessErrorException;

/** @brief Python type for opaque Erlang PIDs */
typedef struct { PyObject_HEAD; ErlNifPid pid; } ErlangPidObject;
extern PyTypeObject ErlangPidType;

/** @brief Cached numpy.ndarray type for fast isinstance checks (NULL if numpy unavailable) */
extern PyObject *g_numpy_ndarray_type;

Expand Down
39 changes: 39 additions & 0 deletions docs/type-conversion.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ When calling Python functions or evaluating expressions, Erlang values are autom
| `list()` | `list` | Recursively converted |
| `tuple()` | `tuple` | Recursively converted |
| `map()` | `dict` | Keys and values recursively converted |
| `pid()` | `erlang.Pid` | Opaque wrapper, round-trips back to Erlang PID |

### Examples

Expand Down Expand Up @@ -74,6 +75,7 @@ Return values from Python are converted back to Erlang:
| `list` | `list()` | Recursively converted |
| `tuple` | `tuple()` | Recursively converted |
| `dict` | `map()` | Keys and values recursively converted |
| `erlang.Pid` | `pid()` | Round-trips back to the original Erlang PID |
| generator | internal | Used with streaming functions |

### Examples
Expand Down Expand Up @@ -114,6 +116,43 @@ Return values from Python are converted back to Erlang:
{ok, #{<<"a">> := 1, <<"b">> := 2}} = py:eval(<<"{'a': 1, 'b': 2}">>).
```

### Process Identifiers (PIDs)

Erlang PIDs are converted to opaque `erlang.Pid` objects in Python. These can be
passed back to Erlang (where they become real PIDs again) or used with `erlang.send()`:

```erlang
%% Pass self() to Python - arrives as erlang.Pid
{ok, Pid} = py:call(mymod, round_trip_pid, [self()]).
%% Pid =:= self()

%% Python can send messages directly to Erlang processes
ok = py:exec(<<"
import erlang
def notify(pid, data):
erlang.send(pid, ('notification', data))
">>).
```

```python
import erlang

def forward_to(pid, message):
"""Send a message to an Erlang process."""
erlang.send(pid, message)
```

`erlang.Pid` objects support equality and hashing, so they can be compared and
used as dict keys or in sets:

```python
pid_a == pid_b # True if both wrap the same Erlang PID
{pid: "value"} # Works as a dict key
pid in seen_pids # Works in sets
```

Sending to a process that has already exited raises `erlang.ProcessError`.

## Special Cases

### NumPy Arrays
Expand Down
Loading