APPLICATION DATABASERelationships & metrics
- customer_id
- plan
- orders
- events
Piipe gives every application one trusted, purpose-bound home for personal information. Your business systems keep doing joins, transactions, segmentation and analytics; Piipe protects the fields that directly identify a person.
APPLICATION DATABASERelationships & metrics
PIIPEProtected identity
Query business data first. Retrieve only the approved PII fields for the final bounded result set.
Piipe does not replace PostgreSQL, MySQL, SQL Server, MongoDB or your warehouse. It gives direct identifiers and sensitive personal fields a dedicated protection boundary. A stable application-owned customer ID preserves the business relationship across both systems.
Use the authenticated JSON HTTPS API from trusted services. Keep credentials out of browsers and request only the customer identity fields each approved workflow needs. Lower-level packet tooling is reserved for controlled appliance integrations.
Make a non-PII UUID the primary key for customers, orders, events and joins.
Link business facts to an organisation-scoped pointer, never to the direct identity.
Resolve the minimum field list at the final step of an approved workflow.
Normal server applications authenticate over HTTPS with the opaque Piipe organisation ID, App ID and Client Secret shown when the application is activated. The caller never supplies a tenant slot or username.
PIIPE_URL="https://api.piipe.io"
PIIPE_ORGANIZATION_ID="org_your_opaque_piipe_id"
PIIPE_APP_ID="42"
PIIPE_CLIENT_SECRET="read-from-your-secret-manager"
# POST these three credential values to /api/login.
# Piipe returns a short-lived opaque session for the fixed piipe-api principal.
Store the opaque organisation ID, App ID and Client Secret in your server-side secrets manager.
Connect to api.piipe.io with your runtime’s normal trusted certificate store. Never disable hostname or certificate checks.
Call POST /api/login. Piipe binds the session to the fixed piipe-api application principal.
Prefer ["name", "email"] over "all" and retrieve only what the approved workflow needs.
Store values that directly identify, contact, locate or reveal a sensitive fact about a person. Keep operational relationships and low-risk business facts in the system designed to query them.
emailphonenameaddressdobdocument
full_name maps to name; date_of_birth maps to dob. A bounded Piipe request supports up to four named fields and 44 combined UTF-8 value bytes.
Move foreign keys away from email or phone and onto a random internal customer ID. Link the customer profile to its Piipe reference. Orders, events, subscriptions and reporting continue to join through the internal ID.
create table customers (
id uuid primary key,
workspace_id uuid not null,
piipe_ref text not null,
piipe_state text not null,
plan_id bigint,
account_status text not null,
age_band text,
created_at timestamptz not null,
unique (workspace_id, piipe_ref)
);
create table orders (
id uuid primary key,
customer_id uuid references customers(id),
product_id bigint not null,
total numeric(12, 2) not null,
created_at timestamptz not null
);
Your relational keyStable random UUID used by foreign keys and application joins.
Your protected-data pointerTenant-scoped reference used only when an approved workflow calls Piipe.
Your destination-scoped aliasSeparate pseudonymous ID for each analytics destination.
| Existing requirement | Piipe pattern |
|---|---|
| Join orders to customers | Join on random customers.id. No PII lookup is needed. |
| Find an exact email or phone | Use an approved keyed lookup index or identity provider to resolve the random customer ID, then retrieve the authorised fields from Piipe. |
| Build a campaign segment | Segment on business facts first; retrieve approved delivery fields just in time. |
| Group by age or geography | Store a justified coarse value such as age_band or broad region, not raw DOB or address. |
| Sort or fuzzy-search a surname | Requires an explicitly reviewed derived index or transient sorting of a small authorised result set. |
Piipe does exact subject or token lookup and selected-field retrieval. It does not expose arbitrary SQL, range filters, substring search, fuzzy matching, joins or analytical aggregation over raw PII. Most business queries remain unchanged; identity-centric search must use the Piipe path.
It derives a deterministic, unkeyed 32-bit token. Tokens can collide and low-entropy values such as emails or phone numbers may be guessed. It is not encryption, anonymisation or a secret blind index. Prefer an application-owned random customer reference as the primary linkage; use a separately reviewed keyed index or identity provider when exact identity lookup is essential.
Funnels, cohorts, retention, product usage, lifetime value and churn models rarely need a name or email. Keep the event facts pseudonymous and resolve identity only for an approved final-mile action.
Use orders, subscriptions, events, scores and approved coarse dimensions.
Confirm the workflow and exact Piipe fields permitted for this action.
Resolve the bounded final set, use it transiently, then discard the PII.
Event type, product, timestamp, amount, plan, segment membership and destination-scoped pseudonymous IDs.
Name, email, phone, exact address, exact DOB, Piipe responses or a reusable cross-platform Piipe reference.
The application database remains pseudonymous personal data, not anonymous data. It still needs access controls, retention limits and privacy governance.
Generate a random customer IDNever derive it from an email, phone number or unkeyed hash.
Validate and minimise PIICollect only fields the workflow genuinely needs.
Write identity to PiipeBind tenant, application, purpose and an exact field set.
Store the reference in SQLUse piipe_state = pending|active|error for reconciliation.
Query business data firstProduce a bounded set of customer IDs with ordinary SQL.
Authorise the declared purposeDo not treat possession of a reference as permission.
Request explicit Piipe fieldsRetrieve only what the final workflow needs.
Use transientlyKeep PII out of logs, URLs, caches, analytics events and browser storage.
Validate the new field valueNormalise exact lookup subjects consistently.
Update Piipe firstThe replacement follows Piipe's journaled commit path.
Reconcile ambiguous outcomesA lost response may follow a successful mutation. Read state before retrying.
Leave relational joins untouchedThe random customer ID remains stable when identity fields change.
Mark deletion pendingStop new workflows that consume the customer's PII.
Request the Piipe tombstoneReconcile until normal lookup confirms the identity is unavailable.
Clear secondary copiesRemove PII from caches, search indexes, queues, exports and logs.
Retain only justified factsLogical deletion is not a claim of physical media erasure.
Normal server applications should use POST /api/login and the JSON HTTPS endpoints below. This internal packet example is deliberately separate: its numeric tenant_id parameter is a Piipe hardware slot, not a public organisation identifier, and it must never be sent to /api/login.
import json
import os
from piipe import (
HttpsTransport,
PiipeApplicationClient,
PiipeConfig,
)
packet_key = bytes.fromhex(os.environ["PIIPE_PACKET_AUTH_KEY_HEX"])
with HttpsTransport(
os.environ.get("PIIPE_HOST", "piipe.example.com"),
port=int(os.environ.get("PIIPE_PORT", "443")),
ca_file=os.environ.get("PIIPE_CA_FILE") or None,
) as transport:
piipe = PiipeApplicationClient.from_transport(
transport,
PiipeConfig(
# Internal hardware slot only; never send this to /api/login.
tenant_id=int(os.environ["PIIPE_INTERNAL_HARDWARE_SLOT"]),
application_id=int(os.environ["PIIPE_APP_ID"]),
purpose_code=int(os.environ["PIIPE_PURPOSE_CODE"]),
session_id=int(os.environ["PIIPE_SESSION_ID"]),
packet_auth_key=packet_key,
initial_request_id=int(os.environ["PIIPE_REQUEST_ID"], 0),
initial_sequence=int(os.environ["PIIPE_SEQUENCE"], 0),
),
)
response = piipe.read_subject_fields(
"crm_customer_12345",
["email", "phone"],
subject_type="customer_id",
)
# Log decision metadata, never the protected response value.
print(json.dumps({
"status": response.status_name,
"decision": response.decision_name,
"request_id": response.request_id,
"audit_id": response.audit_id,
}, indent=2))
Persist the next request and sequence values before use, or provision a fresh session for each new process. Never restart a used session at one.
The SDK verifies the certificate and hostname and requires TLS 1.3.
The SDK validates session state before protected customer identity is changed.
This binary transport is for controlled Piipe appliance integrations. Normal server applications use the JSON HTTPS login and operation endpoints documented in the next section.
https://<piipe-host>:<port>/api/v1/packet
authenticated · binary · no-store
Machine-readable transport envelopeOpenAPI 3.1 · Piipe API v1
| Operation | Purpose | Piipe control |
|---|---|---|
search_record / search_subject | Exact existence lookup without returning PII | Search |
read_fields / read_subject_fields | Retrieve an explicit approved field set | Retrieve |
write_record / write_field_record | Create a protected value or named identity field set | Add |
write_subject_record / write_subject_field_record | Create using a tenant-scoped subject lookup | Add |
update_record / update_field_record | Journal and commit an intentional replacement | Edit |
delete_record / delete_subject_record | Create a logical tombstone | Delete |
verify_attribute | Verify a field claim without returning identity data | Retrieve policy |
detokenise_value / tokenise_value | Compatibility wrappers through the product policy path | Policy dependent |
ping / get_version | Health and compatibility checks | Status |
The internal packet SDK returns a typed response containing operation, status, request and audit identifiers, decision, timing, CRC state and the policy-filtered value. Creates may also return an existing token for an idempotent duplicate.
status_nameoperation_namerequest_idaudit_iddecision_namecyclesmasked_valuecrc_okduplicateexisting_token
Use the authenticated JSON HTTPS API from a trusted server environment. Keep credentials server-side, use finite timeouts, leave TLS verification enabled and close sessions when the workflow ends.
Never place Piipe credentials in browser code. The HTTPS examples use standard server libraries and request only the identity fields the workflow needs.
Use the runtime’s server-side HTTPS client; the examples share the same authenticated login and explicit-field read pattern.
No package installBuilt-in fetch; place the fragment in an ES module such as client.mjs.Standard libraryUse net/http, encoding/json, context and time.JDK + JacksonUse java.net.http; add your organisation’s approved Jackson Databind release for JSON parsing.No package installBuilt-in HttpClient and System.Text.Json.ext-curl + ext-jsonEnable the standard cURL and JSON extensions in the server runtime.Standard libraryUses net/http, json, uri and openssl.| Endpoint | Purpose | Session |
|---|---|---|
POST /api/login | Exchange organization_id, app_id and client_secret for an opaque session bound to the fixed piipe-api application principal. | None |
POST /api/subject-token | Search an exact tenant-scoped subject without returning PII. | tenant_session |
POST /api/write | Add supported named fields for a subject. | tenant_session |
POST /api/read | Retrieve the explicitly requested named fields. | tenant_session |
POST /api/update | Replace named field values through the edit path. | tenant_session |
POST /api/delete | Create a logical tombstone for the subject. | tenant_session |
POST /api/logout | Invalidate the Piipe session. | tenant_session |
/api/login does not accept a username, user ID, numeric tenant slot, role or purpose. Piipe resolves the opaque organisation and returns the session as the organisation’s fixed piipe-api application principal.
{
"tenant_session": "opaque-session",
"lookup": "subject",
"subject": "crm_customer_12345",
"subject_type": "customer_id",
"record_mode": "fields",
"field_data": {
"email": "sample@example.test",
"phone": "+61000000000"
}
}
{
"tenant_session": "opaque-session",
"lookup": "subject",
"subject": "crm_customer_12345",
"subject_type": "customer_id",
"record_mode": "fields",
"fields": "email,phone"
}
const base = process.env.PIIPE_URL;
async function post(path, body) {
const response = await fetch(base + path, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
signal: AbortSignal.timeout(10_000),
});
const data = await response.json();
if (!response.ok || !data.ok) throw new Error(data.error);
return data;
}
const login = await post("/api/login", {
organization_id: process.env.PIIPE_ORGANIZATION_ID,
app_id: Number(process.env.PIIPE_APP_ID),
client_secret: process.env.PIIPE_CLIENT_SECRET,
});
const identity = await post("/api/read", {
tenant_session: login.session,
lookup: "subject",
subject: "crm_customer_12345",
subject_type: "customer_id",
record_mode: "fields",
fields: "email,phone",
});
console.log({
ok: identity.ok,
status: identity.response?.status,
request_id: identity.response?.request_id,
audit_id: identity.response?.audit_id,
});
func post(ctx context.Context, base, path string, input, output any) error {
body, err := json.Marshal(input)
if err != nil { return err }
request, err := http.NewRequestWithContext(ctx, http.MethodPost, base+path, bytes.NewReader(body))
if err != nil { return err }
request.Header.Set("Content-Type", "application/json")
client := &http.Client{Timeout: 10 * time.Second}
response, err := client.Do(request)
if err != nil { return err }
defer response.Body.Close()
payload, err := io.ReadAll(response.Body)
if err != nil { return err }
var envelope map[string]any
if err := json.Unmarshal(payload, &envelope); err != nil {
return err
}
if response.StatusCode/100 != 2 || envelope["ok"] != true {
return fmt.Errorf("Piipe request rejected: %v", envelope["error"])
}
return json.Unmarshal(payload, output)
}
// Call /api/login with organization_id, app_id and client_secret.
// Piipe returns login.session bound to the fixed piipe-api application principal.
static JsonNode post(String base, String path, Object body)
throws Exception {
ObjectMapper json = new ObjectMapper();
HttpRequest request = HttpRequest.newBuilder(URI.create(base + path))
.timeout(Duration.ofSeconds(10))
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(
json.writeValueAsString(body)))
.build();
HttpClient http = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(5))
.build();
HttpResponse<String> response = http
.send(request, HttpResponse.BodyHandlers.ofString());
JsonNode result = json.readTree(response.body());
if (response.statusCode() / 100 != 2 ||
!result.path("ok").asBoolean()) {
throw new IllegalStateException(
result.path("error").asText());
}
return result;
}
using System.Net.Http.Json;
using System.Text.Json;
using var http = new HttpClient {
BaseAddress = new Uri(
Environment.GetEnvironmentVariable("PIIPE_URL")!),
Timeout = TimeSpan.FromSeconds(10),
};
async Task<JsonElement> Post(string path, object body) {
using var response = await http.PostAsJsonAsync(path, body);
var data = await response.Content
.ReadFromJsonAsync<JsonElement>();
if (!response.IsSuccessStatusCode ||
!data.GetProperty("ok").GetBoolean()) {
throw new Exception(data.GetProperty("error").GetString());
}
return data;
}
function piipe_post(string $base, string $path, array $body): array {
$curl = curl_init($base . $path);
curl_setopt_array($curl, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => ['Content-Type: application/json'],
CURLOPT_POSTFIELDS => json_encode($body, JSON_THROW_ON_ERROR),
CURLOPT_CONNECTTIMEOUT => 5,
CURLOPT_TIMEOUT => 10,
]);
$raw = curl_exec($curl);
$error = curl_error($curl);
$status = curl_getinfo($curl, CURLINFO_RESPONSE_CODE);
curl_close($curl);
if ($raw === false) {
throw new RuntimeException($error);
}
$data = json_decode($raw, true, flags: JSON_THROW_ON_ERROR);
if ($status < 200 || $status >= 300 || empty($data['ok'])) {
throw new RuntimeException($data['error'] ?? "HTTP $status");
}
return $data;
}
require "json"
require "net/http"
require "openssl"
require "uri"
def piipe_post(base, path, body)
uri = URI.join("#{base}/", path.delete_prefix("/"))
request = Net::HTTP::Post.new(uri)
request["Content-Type"] = "application/json"
request.body = JSON.generate(body)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = uri.scheme == "https"
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.open_timeout = 5
http.read_timeout = 10
response = http.request(request)
data = JSON.parse(response.body)
unless response.is_a?(Net::HTTPSuccess) && data["ok"]
raise(data["error"] || "HTTP #{response.code}")
end
data
end
BASE="https://api.piipe.io"
LOGIN="$(curl --fail-with-body --silent --show-error \
--connect-timeout 5 --max-time 10 \
-H 'Content-Type: application/json' \
-d '{
\"organization_id\": \"org_your_opaque_piipe_id\",
\"app_id\": 42,
\"client_secret\": \"read-from-secret-store\"
}' \"$BASE/api/login\")"
SESSION="$(printf '%s' \"$LOGIN\" | jq -er '.session')"
curl --fail-with-body --silent --show-error \
--connect-timeout 5 --max-time 10 \
-H 'Content-Type: application/json' \
-d "{
\"tenant_session\": \"$SESSION\",
\"lookup\": \"subject\",
\"subject\": \"crm_customer_12345\",
\"subject_type\": \"customer_id\",
\"record_mode\": \"fields\",
\"fields\": \"email,phone\"
}" \
"$BASE/api/read" |
jq '{ok, status: .response.status,
request_id: .response.request_id,
audit_id: .response.audit_id}'
After the final request, call POST /api/logout with {"tenant_session":"…"} from a finally, defer, ensure or equivalent cleanup path.
Verify HTTPS; never bypass it.api.piipe.io uses a publicly trusted certificate. Do not use curl -k, verify=False or a process-wide “accept all certificates” callback.
The HTTPS examples intentionally do not retry a POST after a transport failure. Piipe may have committed the operation before the connection was lost. Query or reconcile state before deciding whether another mutation is safe. The typed exceptions below belong to the advanced internal packet SDK; JSON clients receive the corresponding error envelope.
PipeClosedErrorYour organisation’s Piipe Controls currently block this action. The exception carries the verified response.
HardwareEpochErrorThe protected session changed before a mutation completed. Refresh the session state before retrying.
PacketErrorThe fixed response failed size, version, operation, request, integrity or proof validation.
TransportErrorTLS trust, hostname, HTTP framing, timeout or connection validation failed.
PipeControlConflictErrorA Piipe Control changed after the request began. Refresh the control state before retrying.
PiipeErrorThe SDK rejected configuration or an operation returned a non-success status.
OKNOT_FOUNDDENIEDBAD_PACKETBAD_AUTHREPLAYOVERLOADEDPIPE_CLOSEDCONFLICTEPOCH_MISMATCHCreate your Piipe account, open your private workspace and connect your first application.