# 📋 MS Field Service Extraction Plan

> **Target:** Dynamics 365 Field Service Mobile App (Android)  
> **Device:** Rooted Pixel 4a — Magisk v30.7, su confirmed  
> **Goal:** Read-only extraction — API surface mapping, data catalog, auth mechanism — for canteen connector  
> **Status:** Draft  
> **Last updated:** 2026-05-26

---

## Table of Contents

1. [Prerequisites & Environment](#1-prerequisites--environment)
2. [APK Decompilation & Static Analysis](#2-apk-decompilation--static-analysis)
3. [Frida Hooking Strategy](#3-frida-hooking-strategy)
4. [Traffic Capture Methodology](#4-traffic-capture-methodology)
5. [Authentication Analysis](#5-authentication-analysis)
6. [Data Dump Procedure](#6-data-dump-procedure)
7. [Data Catalog Template](#7-data-catalog-template)
8. [Risk & Mitigations](#8-risk--mitigations)
9. [Task Dependencies & Execution Order](#9-task-dependencies--execution-order)
10. [Tools & Versions](#10-tools--versions)

---

## 1. Prerequisites & Environment

### 1.1 Android Device Setup

| Item | Status |
|------|--------|
| Rooted Pixel 4a (Magisk v30.7) | ✅ Confirmed |
| ADB + USB debugging enabled | ⚠️ Verify |
| Device developer options unlocked | ⚠️ Verify |
| `adb root` working | ⚠️ Verify |
| MagiskHide / Zygisk enabled for target app | ⚠️ Configure |
| Frida server running on device | ⚠️ Install |
| mitmproxy CA installed in system trust store | ⚠️ Install |
| Burp / mitmproxy accessible from device | ⚠️ Set up proxy |

```bash
# Verify device connection
adb devices -l
adb shell "su -c id"                    # Confirm root
adb shell "getprop ro.build.version.sdk"
adb shell "pm list packages | grep -i fieldservice"

# Push Frida server (match device arch)
adb shell "uname -m"                    # Usually aarch64
adb push frida-server-16.6.6-android-arm64 /data/local/tmp/
adb shell "chmod 755 /data/local/tmp/frida-server-16.6.6-android-arm64"
adb shell "su -c /data/local/tmp/frida-server-16.6.6-android-arm64 -D &"
```

### 1.2 Host Workstation Setup

```bash
# Tools (install if missing)
pip install frida-tools mitmproxy apkleaks
sudo apt install apktool jadx dex2jar adb
npm install -g apk-mitm                      # Auto cert-pinning patch
```

### 1.3 Target Identification

Determine the exact package name and app variant:

```bash
# List Dynamics/Field Service related packages
adb shell "pm list packages | grep -iE 'microsoft|dynamics|fieldservice|crm|mscrm'"
# Get APK path
adb shell "pm path com.microsoft.<package>"   # After identifying package name
adb pull /data/app/com.microsoft.<package>/base.apk
```

Likely package name: `com.microsoft.dynamics.fieldservice` or `com.microsoft.crm.phone`

---

## 2. APK Decompilation & Static Analysis

### 2.1 APK Acquisition

```bash
# Pull the APK from device (after identifying package)
adb shell pm path <package> | cut -d: -f2 | xargs adb pull

# If app uses split APKs, pull all:
adb shell pm path <package> | cut -d: -f2 | while read f; do
  adb pull "$f"
done
```

### 2.2 Decompilation Pipeline

```bash
# Step 1: apktool — extract resources, AndroidManifest, smali
apktool d -f -o ./apktool-out base.apk

# Step 2: jadx — decompile to Java source
jadx -d ./jadx-out --show-bad-code base.apk

# Step 3: APKLeaks — automated secrets enumeration
apkleaks -f base.apk -o ./apkleaks-results.txt

# Step 4: Extract native libs (for pinned libs / hooking targets)
mkdir -p ./native-libs
find ./apktool-out -name "*.so" -exec cp {} ./native-libs/ \;
```

### 2.3 Key Artifacts to Extract

| Artifact | Location (jadx-out) | Why |
|----------|---------------------|-----|
| `AndroidManifest.xml` | Root of apktool-out | Permissions, auth intents, network config |
| `network_security_config.xml` | `res/xml/` | Certificate pinning / trust config |
| MSAL/Azure AD config | Search for `msal_`, `client_id`, `authority` | Auth endpoints |
| API base URLs | Search for `api.*dynamics`, `dataverse`, `crm` | API surface |
| Retrofit/OkHttp clients | Search for `Retrofit`, `OkHttpClient` | Network layer |
| ProGuard mappings | `mapping.txt` if present | Obfuscation recovery |

### 2.4 Static Analysis Commands

```bash
# Search for API endpoint strings
grep -rPn '"https?://[^"]*' ./jadx-out/ | grep -iv 'google\|microsoft.com/common' | head -100

# Search for Dataverse/CRM endpoints
grep -rPn '(dataverse|crm\d?\.dynamics|api\.dynamics)' ./jadx-out/ --include='*.java'

# Search for client IDs and auth config
grep -rPn '(client_id|ClientId|applicationId|msal|authority|tenant)' ./jadx-out/ --include='*.java'

# Search for certificate pinning implementations
grep -rPn '(certificatePinner|CertificatePinner|pins|PinningTrustManager)' ./jadx-out/ --include='*.java'

# List permissions requested
grep 'uses-permission' ./apktool-out/AndroidManifest.xml
```

### 2.5 Obfuscation Assessment

MS Field Service uses ProGuard/R8 obfuscation. Expect:
- Classes/methods renamed to `a`, `b`, `c` etc.
- Strings may be obfuscated
- Network layer wrapped in internal abstractions

**Mitigation:** Use Frida to hook known libraries (OkHttp, MSAL) at the Java/JNI boundary rather than trying to follow obfuscated call paths.

---

## 3. Frida Hooking Strategy

### 3.1 Frida Server Setup

```bash
# On device as root
/data/local/tmp/frida-server-16.6.6-android-arm64 &

# Verify
frida-ps -U | grep frida-server
```

### 3.2 Frida Scripts — In Development Order

#### Phase 1: Identify the Network Layer (Hook OkHttp/Retrofit)

Hook **all** OkHttp and Retrofit calls to see API surface at runtime:

```javascript
// hook-okhttp.js
Java.perform(function() {
    // Hook OkHttp Client
    var OkHttpClient = Java.use('okhttp3.OkHttpClient');
    OkHttpClient.newCall.overload('okhttp3.Request').implementation = function(request) {
        console.log('[NET] ' + request.method() + ' ' + request.url().toString());
        console.log('[NET] Headers: ' + JSON.stringify(request.headers().toMultimap()));
        return this.newCall(request);
    };

    // Hook OkHttp Interceptor
    var HttpURLConnection = Java.use('java.net.HttpURLConnection');
    HttpURLConnection.connect.implementation = function() {
        console.log('[HUC] URL: ' + this.getURL());
        return this.connect();
    };
});
```

#### Phase 2: Intercept Auth Token Acquisition (MSAL / ADAL)

```javascript
// hook-msal.js
Java.perform(function() {
    // Microsoft Authentication Library
    var PCA = Java.use('com.microsoft.identity.client.PublicClientApplication');
    PCA.acquireToken.overloads.forEach(function(overload) {
        overload.implementation = function() {
            console.log('[MSAL] acquireToken called');
            // Log the arguments
            for (var i = 0; i < arguments.length; i++) {
                console.log('[MSAL] arg[' + i + ']: ' + arguments[i]);
            }
            // Also hook the callback to capture result
            var callback = arguments[arguments.length - 1];
            if (callback) {
                var originalOnSuccess = callback.onSuccess;
                callback.onSuccess = function(authResult) {
                    console.log('[MSAL] SUCCESS - Access Token: ' + authResult.getAccessToken());
                    console.log('[MSAL] Account: ' + authResult.getAccount().getUsername());
                    // SAVE THIS: dump to file
                    send({type: 'token', token: authResult.getAccessToken(),
                          account: authResult.getAccount().getUsername()});
                    return originalOnSuccess.call(this, authResult);
                };
            }
            return overload.apply(this, arguments);
        };
    });

    // Alternative: Hook MSAL's ResultFuture or AuthenticationCallback
    try {
        var AuthCallback = Java.use('com.microsoft.identity.client.IAuthenticationResult');
        // Hooks for getAccessToken in results
    } catch(e) { console.log('[MSAL] IAuthenticationResult not found, trying alternative...'); }

    // Fallback: Hook Android AccountManager for auth tokens
    var AccountManager = Java.use('android.accounts.AccountManager');
    AccountManager.getAuthToken.overloads.forEach(function(overload) {
        overload.implementation = function() {
            console.log('[AM] getAuthToken called');
            var result = overload.apply(this, arguments);
            console.log('[AM] getAuthToken result: ' + result);
            return result;
        };
    });
});
```

#### Phase 3: Data Interception — Hook JSON Response Parsing

```javascript
// hook-dataverse.js
Java.perform(function() {
    // Hook HTTP response body reading
    var Buffer = Java.use('okio.Buffer');
    Buffer.writeTo.overload('java.io.OutputStream').implementation = function(sink) {
        // Capture response body before it's consumed
        var snapshot = this.clone();
        var bytes = snapshot.readByteArray();
        var body = bytes !== null ? String.fromCharCode.apply(null, bytes) : '';
        if (body.length > 20 && body.length < 50000) {
            console.log('[BODY] ' + body);
            send({type: 'response', data: body});
        }
        return this.writeTo(sink);
    };

    // Hook specific DataVerse API patterns
    var OkHttpResponse = Java.use('okhttp3.Response');
    OkHttpResponse.body.implementation = function() {
        var body = this.body();
        var url = this.request().url().toString();
        if (url.indexOf('api/data/v') > -1 || url.indexOf('api/fieldservice') > -1) {
            console.log('[DV] Response URL: ' + url);
            console.log('[DV] Code: ' + this.code());
        }
        return body;
    };
});
```

#### Phase 4: SQLite / Local DB Dump

```javascript
// hook-sqlite.js
Java.perform(function() {
    var SQLiteDatabase = Java.use('android.database.sqlite.SQLiteDatabase');
    SQLiteDatabase.rawQuery.overload('java.lang.String', '[Ljava.lang.String;').implementation = function(sql, args) {
        console.log('[SQL] ' + sql);
        if (args) {
            for (var i = 0; i < args.length; i++) {
                console.log('[SQL] arg[' + i + '] = ' + args[i]);
            }
        }
        return this.rawQuery(sql, args);
    };

    SQLiteDatabase.execSQL.overload('java.lang.String').implementation = function(sql) {
        console.log('[SQL-EXEC] ' + sql);
        return this.execSQL(sql);
    };
});
```

### 3.3 Running Frida Scripts

```bash
# Basic hook (no spawn gadget — use root)
frida -U -f <package> -l hook-okhttp.js --no-pause

# With certificate unpinning on this run
frida -U -f <package> -l hook-okhttp.js -l frida-cert-unpin.js --no-pause

# Attach to running app (after login flow)
frida -U <package> -l hook-msal.js
```

### 3.4 Frida Gadget (Alternative — No Root Needed)

If the app detects Frida via port scanning:

```bash
# Inject frida-gadget into APK
apk-mitm <base.apk>         # Also does cert unpinning!
# Or manually:
apktool d base.apk
cp frida-gadget-16.6.6-android-arm64.so ./apktool-out/lib/arm64-v8a/libfrida-gadget.so
# Then edit smali to load it (or use LD_PRELOAD approach)
apktool b -o patched.apk
# Sign and install
adb install patched.apk
```

---

## 4. Traffic Capture Methodology

### 4.1 Proxy Setup

#### Option A: mitmproxy (Recommended for API Recon)

```bash
# Start mitmproxy on host (reverse proxy mode)
mitmweb --listen-host 0.0.0.0 --listen-port 8080
```

Device config via `adb`:
```bash
# Set HTTP proxy (requires Wi-Fi connection on device)
adb shell settings put global http_proxy <host-ip>:8080
adb shell settings put global global_http_proxy_host <host-ip>
adb shell settings put global global_http_proxy_port 8080
```

#### Option B: Transparent proxying via iptables (for apps that ignore proxy)

```bash
# On device (root):
iptables -t nat -A OUTPUT -p tcp --dport 443 -j DNAT --to-destination <host-ip>:8080
iptables -t nat -A OUTPUT -p tcp --dport 80  -j DNAT --to-destination <host-ip>:8080

# Redirect packets to local proxy
iptables -t nat -A OUTPUT -p tcp --dport 443 ! -d 127.0.0.1 -j REDIRECT --to-port 8080
```

#### Option C: PCAP-level capture (last resort)

```bash
# tcpdump on device (root)
adb shell "su -c tcpdump -i any -s 0 -w /sdcard/capture.pcap host <host-ip> &"
# After session, pull and analyze with Wireshark
adb pull /sdcard/capture.pcap
```

### 4.2 Certificate Handling

#### Stage 1: System CA Installation (mitmproxy)

```bash
# Generate mitmproxy CA
mitmproxy --listen-port 8080   # First run generates ~/.mitmproxy/mitmproxy-ca-cert.cer

# Push to device system trust store (root):
adb push ~/.mitmproxy/mitmproxy-ca-cert.cer /sdcard/
adb shell "su -c cp /sdcard/mitmproxy-ca-cert.cer /system/etc/security/cacerts/87c0af38.0"
adb shell "su -c chmod 644 /system/etc/security/cacerts/87c0af38.0"
adb shell "su -c reboot"       # Some devices need reboot to pick up CA
```

If `/system` is read-only (Pixel 4a with Magisk):
```bash
# Use Magisk module "Move Certificates" or manual overlay:
adb shell "su -c mkdir -p /data/adb/modules/mitmproxy-ca/system/etc/security/cacerts"
adb shell "su -c cp /sdcard/mitmproxy-ca-cert.cer /data/adb/modules/mitmproxy-ca/system/etc/security/cacerts/87c0af38.0"
adb shell "su -c chmod 644 /data/adb/modules/mitmproxy-ca/system/etc/security/cacerts/87c0af38.0"
adb shell "su -c reboot"
```

#### Stage 2: Certificate Pinning Bypass (if needed)

The MS Field Service app likely uses **certificate pinning** via:
- OkHttp `CertificatePinner` (Java)
- Android Network Security Config (XML)
- Native OpenSSL pinning (NDK library)

**Bypass approaches (in order):**

1. **`apk-mitm` tool** — Auto-patches network security config:
   ```bash
   npm install -g apk-mitm
   apk-mitm base.apk
   # Creates base-patched.apk with cert pinning disabled
   ```

2. **Universal Frida unpin script:**
   ```bash
   # Use the well-known frida-scripts repo
   frida -U -f <package> -l frida-multiple-unpinning.js --no-pause
   ```

3. **If using OkHttp CertificatePinner:**
   ```javascript
   Java.perform(function() {
       var CertificatePinner = Java.use('okhttp3.CertificatePinner');
       CertificatePinner.check.overload('java.lang.String', 'java.util.List').implementation = function() {
           console.log('[UNPIN] Bypassing certificate check');
           return;  // Skip pin verification
       };
   });
   ```

4. **Custom TrustManager injection:**
   ```javascript
   Java.perform(function() {
       var TrustManager = Java.registerClass({
           name: 'com.example.TrustAllManager',
           implements: [Java.use('javax.net.ssl.X509TrustManager')],
           methods: {
               checkClientTrusted: function(chain, authType) {},
               checkServerTrusted: function(chain, authType) {},
               getAcceptedIssuers: function() { return []; }
           }
       });
       // Then replace SSLContext default TrustManager
   });
   ```

### 4.3 Traffic Analysis Workflow

1. Start mitmproxy in recording mode:
   ```bash
   mitmproxy --listen-port 8080 --save-stream-file ./capture.flow
   ```

2. Launch app, perform typical user actions (login, navigate, sync)

3. Export captured requests:
   ```bash
   # Extract URLs, methods, status codes
   mitmproxy --rfile ./capture.flow --export-format json > ./capture.json

   # Extract only API calls (filter out CDN/telemetry)
   cat capture.json | jq 'select(.request.url | contains("dynamics") or contains("api/data"))' > ./api-capture.json
   ```

4. **Key endpoints to look for:**

| Pattern | Likely Purpose |
|---------|---------------|
| `*.crm*.dynamics.com/api/data/v9.2/*` | Dataverse WebAPI |
| `*.dynamics.com/api/fieldservice/v1/*` | Field Service specific APIs |
| `*.dynamics.com/api/fieldservice/v2/*` | FSO v2 endpoints |
| `*.dynamics.com/api/data/v9.2/msdyn_*` | Field Service custom entities |
| `*.dynamics.com/api/data/v9.2/bookableresources` | Resource scheduling |
| `*.dynamics.com/api/data/v9.2/workorders` | Work orders |
| `*.crm*.dynamics.com/*/oauth/token` | OAuth token refresh |
| `login.microsoftonline.com/*` | Azure AD auth |
| `*.azureedge.net` | CDN / static assets (skip) |
| `dc.services.visualstudio.com` | Telemetry (skip) |

---

## 5. Authentication Analysis

### 5.1 Dynamics 365 Auth Flow

The Field Service mobile app uses **Azure AD (Entra ID)** OAuth 2.0 with **MSAL** (Microsoft Authentication Library). The typical flow:

```
App → MSAL SDK → login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize
                → User authenticates (interactive)
                → Returns auth code
                → MSAL exchanges for tokens
                → Stores in Android AccountManager / MSAL cache

Access Token → Sent as Bearer token in Authorization header
Refresh Token → Rotated on each refresh (MSAL opaque to app)
```

### 5.2 Token Capture Strategy

#### Via Frida (Runtime):

```javascript
// hook-msal-tokens.js
Java.perform(function() {
    var AuthenticationResult = Java.use('com.microsoft.identity.client.IAuthenticationResult');
    if (AuthenticationResult) {
        AuthenticationResult.getAccessToken.implementation = function() {
            var token = this.getAccessToken();
            console.log('[TOKEN] AccessToken: ' + token);
            send({type: 'access_token', value: token});
            return token;
        };
    }

    // Hook MSAL token cache
    try {
        var TokenCache = Java.use('com.microsoft.identity.client.ITokenCache');
        var AccountCache = Java.use('com.microsoft.identity.client.IAccount');
        // ... cache hooks
    } catch(e) {
        console.log('[TOKEN] ITokenCache not found directly');
    }

    // Hook AccountManager-based token retrieval
    var AccountManager = Java.use('android.accounts.AccountManager');
    AccountManager.peekAuthToken.implementation = function(account, tokenType) {
        var token = this.peekAuthToken(account, tokenType);
        console.log('[AM] peekAuthToken: ' + tokenType + ' => ' + token);
        return token;
    };

    AccountManager.getPassword.implementation = function(account) {
        var pwd = this.getPassword(account);
        console.log('[AM] password for ' + account.name + ': ' + pwd);
        return pwd;
    };
});
```

#### Via mitmproxy (Token refresh interception):

```python
# addons/token-capture.py
from mitmproxy import http
import json

class TokenCapture:
    def response(self, flow: http.HTTPFlow):
        url = flow.request.pretty_url
        # OAuth token endpoint responses
        if 'login.microsoftonline.com' in url and 'oauth2' in url:
            if flow.response.status_code == 200:
                try:
                    body = json.loads(flow.response.text)
                    if 'access_token' in body:
                        print(f"\n===== CAPTURED TOKEN =====")
                        print(f"access_token: {body['access_token'][:50]}...")
                        print(f"token_type: {body.get('token_type')}")
                        print(f"expires_in: {body.get('expires_in')}")
                        print(f"scope: {body.get('scope')}")
                        print(f"resource/id: {body.get('resource') or body.get('id_token')}")
                        print(f"==========================\n")
                        # Log to file
                        with open('/tmp/captured_tokens.log', 'a') as f:
                            f.write(json.dumps({
                                'type': 'oauth_token',
                                'access_token': body['access_token'],
                                'refresh_token': body.get('refresh_token'),
                                'scope': body.get('scope'),
                                'expires_in': body.get('expires_in'),
                                'tenant': url.split('/')[3] if '/login/' in url else None
                            }) + '\n')
                except (json.JSONDecodeError, KeyError):
                    pass

addons = [TokenCapture()]
```

### 5.3 Token Analysis

Once captured, examine the token payload:

```bash
# Decode JWT (access token) header + payload
python3 -c "
import sys, json, base64
token = sys.argv[1]
parts = token.split('.')
for i, part in enumerate(parts[:2]):  # header + payload
    padded = part + '=' * (4 - len(part) % 4)
    decoded = base64.urlsafe_b64decode(padded)
    print(f'--- Part {i} ---')
    print(json.dumps(json.loads(decoded), indent=2))
" <access_token>
```

Key claims to extract:
- `aud` — Audience (should be Dataverse API URI, e.g. `https://<org>.crm.dynamics.com`)
- `iss` — Issuer (Entra tenant ID)
- `scp` — Scopes (what the token allows)
- `appid` — Client ID (identifies the mobile app in Entra)
- `tid` — Tenant ID
- `upn` — User Principal Name
- `exp` / `nbf` — Validity window

### 5.4 Service Principal Approach (Server-to-Server)

For a **canteen connector** that runs server-side, a mobile user token is **not** the right auth mechanism. Instead, use **Application (Service Principal) auth**:

```bash
# 1. Register an application in Entra ID (same tenant as Field Service)
#    - Azure Portal → App registrations → New registration
#    - Name: "Canteen Field Service Connector"
#    - Supported account types: "Accounts in this organizational directory only"
#    - Redirect URI: (not needed for client credentials flow)

# 2. Grant API permissions
#    - Dynamics CRM → `user_impersonation` (delegated)
#    - OR: Create Application User in Power Platform Admin Center

# 3. Create client secret
#    - Azure Portal → Certificates & secrets → New client secret

# 4. Get token via client credentials flow:
curl -X POST https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'client_id={app_id}&client_secret={secret}&grant_type=client_credentials&scope=https://{org}.crm.dynamics.com/.default'

# 5. Use token to call Dataverse APIs:
curl -H "Authorization: Bearer $TOKEN" \
     -H "Accept: application/json" \
     -H "OData-MaxVersion: 4.0" \
     "https://{org}.crm.dynamics.com/api/data/v9.2/msdyn_workorders"
```

### 5.5 Auth Comparison

| Aspect | Captured User Token | Service Principal |
|--------|-------------------|-------------------|
| **Context** | Specific user's session | Application identity |
| **Lifetime** | 60-90 min (refreshable) | Configurable (up to 1yr) |
| **Scope** | What the user can see | Controlled via Application User |
| **Reliability** | Breaks on password change | Stable |
| **Setup complexity** | Requires interactive login | Server-side, automated |
| **Recommended for canteen?** | ❌ No | ✅ Yes |

**Decision:** The canteen connector should use **Service Principal** auth with `client_credentials` flow. The mobile token capture is only for **mapping the API surface** — discovering which endpoints/entities exist and what data they return.

### 5.6 API Permission Mapping

Once we have tokens, we can probe the API:

```bash
# 1. List all entities (WebAPI $metadata)
curl -H "Authorization: Bearer $TOKEN" \
     "https://{org}.crm.dynamics.com/api/data/v9.2/$metadata" -o metadata.xml

# 2. Focus on Field Service entities (msdyn_ prefix)
cat metadata.xml | grep -oP 'EntityType Name="\K[^"]+' | grep -i 'msdyn_' | sort

# 3. Test access to key entities
entities=(
  msdyn_workorders
  msdyn_bookableresources
  msdyn_actuals
  msdyn_customers
  msdyn_serviceagreements
  msdyn_products
  msdyn_services
  msdyn_fieldservicepricelists
  msdyn_systemusers
  msdyn_bookingheaders
)

for entity in "${entities[@]}"; do
  echo "=== $entity ==="
  curl -s -H "Authorization: Bearer $TOKEN" \
       "https://{org}.crm.dynamics.com/api/data/v9.2/$entity?\$top=1" \
       | python3 -c "import sys,json; d=json.load(sys.stdin); print(list(d.get('value',[{}])[0].keys()) if d.get('value') else d)" 2>&1
done
```

---

## 6. Data Dump Procedure

### 6.1 Sequential Dump Plan

Phase 1 should be **metadata-only** (entity names, field names). Phase 2 samples data. Phase 3 (if needed) extracts all records.

#### Phase 1: Schema Discovery

```bash
# Dump $metadata
curl -o metadata.xml \
     -H "Authorization: Bearer $TOKEN" \
     "https://{org}.crm.dynamics.com/api/data/v9.2/\$metadata"

# Parse metadata to JSON catalog
python3 << 'PYEOF'
import xml.etree.ElementTree as ET
import json, sys

tree = ET.parse('metadata.xml')
ns = {'edm': 'http://docs.oasis-open.org/odata/ns/edm',
      'csdl': 'http://docs.oasis-open.org/odata/ns/edmx'}

catalog = {}

# Find all entity types
root = tree.getroot()
schema = root.find('.//edm:Schema', ns)
if schema is not None:
    for entity in schema.findall('edm:EntityType', ns):
        name = entity.attrib['Name']
        if not name.startswith('msdyn_'):
            continue
        fields = []
        for prop in entity.findall('edm:Property', ns):
            fields.append({
                'name': prop.attrib['Name'],
                'type': prop.attrib.get('Type', 'Edm.String'),
                'nullable': prop.attrib.get('Nullable', 'true'),
                'maxLength': prop.attrib.get('MaxLength', 'N/A')
            })
        for nav in entity.findall('edm:NavigationProperty', ns):
            fields.append({
                'name': nav.attrib['Name'],
                'type': 'Navigation',
                'partner': nav.attrib.get('Partner', ''),
                'ref': nav.attrib.get('Type', '')
            })
        catalog[name] = fields

with open('entity-catalog.json', 'w') as f:
    json.dump(catalog, f, indent=2)

print(f"Catalogued {len(catalog)} entities")
print("Entities:", list(catalog.keys()))
PYEOF
```

#### Phase 2: Data Sampling

```bash
# For each entity, grab one record to see field shapes
for entity in msdyn_workorders msdyn_bookableresources msdyn_actuals msdyn_customers; do
  curl -s -H "Authorization: Bearer $TOKEN" \
       "https://{org}.crm.dynamics.com/api/data/v9.2/$entity?\$top=1&\$select=*" \
       -o "sample-$entity.json"
  echo "Sampled $entity: $(python3 -c "import json; d=json.load(open('sample-$entity.json')); print(f\"{len(d.get('value',[]))} records, {len(list(d.get('value',[{}])[0].keys())) if d.get('value') else 0} fields\")")"
done

# List all relationships / lookup fields
curl -s -H "Authorization: Bearer $TOKEN" \
     "https://{org}.crm.dynamics.com/api/data/v9.2/msdyn_workorders?\$top=1&\$expand=*" \
     -o sample-relationships.json 2>/dev/null || \
echo "Expand all failed — no direct * support. Try named navigations instead."
```

#### Phase 3: Bulk Extract (Conditional — only if full data is needed)

```bash
# Using paging (Dataverse returns max 5000 per page)
EXTRACT_DIR="./data-extract"
mkdir -p "$EXTRACT_DIR"

extract_entity() {
  local entity=$1
  local base_url="https://{org}.crm.dynamics.com/api/data/v9.2/$entity"
  local next_link="$base_url?\$top=5000"
  local page=1

  while [ -n "$next_link" ]; do
    echo "Extracting $entity page $page..."
    curl -s -H "Authorization: Bearer $TOKEN" \
         -H "Prefer: odata.include-annotations=*" \
         "$next_link" > "$EXTRACT_DIR/${entity}_page${page}.json"

    next_link=$(python3 -c "
import json
d=json.load(open('$EXTRACT_DIR/${entity}_page${page}.json'))
if '@odata.nextLink' in d:
    print(d['@odata.nextLink'])
" 2>/dev/null)

    page=$((page + 1))
    # Safety cap
    [ $page -gt 100 ] && break
  done
}

# Run only on entities we know we need
extract_entity msdyn_workorders
extract_entity msdyn_bookableresources
```

### 6.2 Local DB Extraction (Offline Cache)

The app may cache data locally in SQLite:

```bash
# Find app's databases
adb shell "su -c find /data/data/<package>/databases -type f"
adb shell "su -c ls -la /data/data/<package>/databases/"

# Pull databases
adb shell "su -c cp /data/data/<package>/databases/*.db /sdcard/"
adb pull /sdcard/*.db ./app-databases/

# Pull shared preferences (contains cached auth, settings)
adb shell "su -c cp -r /data/data/<package>/shared_prefs /sdcard/"
adb pull /sdcard/shared_prefs ./app-shared-preferences/

# Analyze databases
sqlite3 ./app-databases/<db>.db ".tables"
sqlite3 ./app-databases/<db>.db ".schema"
sqlite3 ./app-databases/<db>.db "SELECT * FROM <table> LIMIT 10;"
```

---

## 7. Data Catalog Template

After schema discovery, document the data catalog in this format:

```markdown
# Data Catalog — MS Field Service

## Entity: msdyn_workorder

| Field | Type | Nullable | Notes |
|-------|------|----------|-------|
| `msdyn_workorderid` | Guid | No | Primary key |
| `msdyn_name` | String(100) | No | Work order title/ID |
| `msdyn_systemstatus` | Picklist | No | Status (e.g. 690970000 = Open) |
| `msdyn_serviceterritory` | Lookup | Yes | Reference to territory |
| `msdyn_serviceaccount` | Customer | Yes | Account being serviced |
| `msdyn_incidenttype` | Lookup | Yes | Type of incident |
| `msdyn_priority` | Picklist | Yes | 1-10 priority |
| `msdyn_substatus` | String(500) | Yes | Sub-status description |
| `createdon` | DateTime | No | Record creation |
| `modifiedon` | DateTime | No | Last modified |

**Navigation properties:**
- `msdyn_msdyn_workorder_msdyn_workorderservicetasks_WorkOrder` → Service Tasks
- `msdyn_msdyn_workorder_msdyn_workorderproducts_WorkOrder` → Products
- `msdyn_msdyn_workorder_msdyn_workorderservices_WorkOrder` → Services
- `msdyn_bookableresource_workorder` → Assigned resource

**API URL:** `/api/data/v9.2/msdyn_workorders`

---

## Entity: msdyn_bookableresource

| Field | Type | Nullable | Notes |
|-------|------|----------|-------|
| `msdyn_bookableresourceid` | Guid | No | Primary key |
| `msdyn_name` | String(100) | No | Resource name |
| `msdyn_resource_type` | Picklist | No | User, Equipment, Facility, Crew |
| `msdyn_user` | Lookup | Yes | Linked systemuser |
| `msdyn_employee_type` | Picklist | Yes | Employee / Contractor |
| `msdyn_displayinschedulingscreen` | Boolean | No | Visibility toggle |

---

## Entity: msdyn_actual

| Field | Type | Nullable | Notes |
|-------|------|----------|-------|
| `msdyn_actualid` | Guid | No | Primary key |
| `msdyn_name` | String(100) | Yes | Description |
| `msdyn_transactiontype` | Picklist | No | Cost, Revenue, etc. |
| `msdyn_amount` | Decimal | No | Monetary amount |
| `msdyn_quantity` | Decimal | No | Units |
| `msdyn_unit` | Lookup | Yes | uom |
| `msdyn_workorder` | Lookup | Yes | Parent work order |

---

## Entity: msdyn_customerasset (Equipment at customer site)

| Field | Type | Nullable | Notes |
|-------|------|----------|-------|
| `msdyn_customerassetid` | Guid | No | Primary key |
| `msdyn_name` | String(100) | No | Asset name/tag |
| `msdyn_serialnumber` | String(100) | Yes | Serial number |
| `msdyn_installedon` | DateTime | Yes | Installation date |
| `msdyn_warrantystart` | DateTime | Yes | Warranty start |
| `msdyn_warrantyend` | DateTime | Yes | Warranty expiry |
| `msdyn_serviceaccount` | Customer | Yes | Where it's installed |
| `msdyn_primaryincidenttype` | Lookup | Yes | Default incident type |

---

## Entity: msdyn_fieldservicesetting (System configuration)

| Field | Type | Nullable | Notes |
|-------|------|----------|-------|
| `msdyn_name` | String(100) | No | Setting name |
| `msdyn_autogenerateworkordernumbers` | Boolean | No | Auto-numbering |
| `msdyn_defaultworkorderserviceretry` | Integer | Yes | Retry count |
| `msdyn_defaultworkordertype` | Lookup | Yes | Default type |
| `msdyn_workorderautoapproval` | Boolean | No | Auto-approval |

```

### Catalog Generation Command

After initial metadata collection, generate the template:

```bash
python3 << 'PYEOF'
import json

with open('entity-catalog.json') as f:
    catalog = json.load(f)

def picklist_label(val, field):
    """Placeholder — real labels come from GlobalOptionSetDefinitions in metadata"""
    return f"Picklist (check $metadata for labels)"

type_map = {
    'Edm.String': lambda f: f"String({f.get('maxLength', 'N/A')})",
    'Edm.Int32': 'Integer',
    'Edm.Int64': 'Long',
    'Edm.Boolean': 'Boolean',
    'Edm.DateTime': 'DateTime',
    'Edm.Decimal': 'Decimal',
    'Edm.Guid': 'Guid',
    'Edm.Double': 'Double',
    'Navigation': 'Navigation',
}

for ename, fields in catalog.items():
    print(f"\n## Entity: {ename}\n")
    print("| Field | Type | Nullable | Notes |")
    print("|-------|------|----------|-------|")
    for f in fields:
        ftype = type_map.get(f.get('type'), f.get('type', 'Unknown'))
        notes = ""
        if f.get('type') == 'Picklist' or 'OptionSet' in str(f.get('type')):
            notes = "Picklist (check $metadata for labels)"
        elif f.get('type') == 'Navigation':
            notes = f"Navigation → {f.get('ref', '?')}"
            ftype = 'Lookup / Nav'

        nullable = f.get('nullable', 'true')
        fname = f'`{f["name"]}`'
        print(f"| {fname} | {ftype} | {nullable} | {notes} |")
    print()
PYEOF
```

---

## 8. Risk & Mitigations

### 8.1 Certificate Pinning

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| OkHttp CertificatePinner | **High** | App won't connect through proxy | Frida `CertificatePinner.check` bypass |
| Android Network Security Config | **High** | Blocks non-system CAs | `apk-mitm` auto-patches or modify `network_security_config.xml` |
| Native (NDK) pinning | Medium | Requires native lib reverse | Hook at `SSL_CTX_set_cert_verify_callback` level |
| Pinning with backup pins | Medium | Pins rotate but still block | Same bypass as OkHttp + monitor for new pins |

**Mitigation plan:**
1. Start with `apk-mitm` (handles ~90% of cases)
2. Add universal Frida unpin script
3. If native pinning suspected, hook `BoringSSL` / `OpenSSL` `SSL_CTX_set_custom_verify`

### 8.2 DPoP (Demonstration of Proof of Possession)

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| DPoP-bound tokens | **Low-Medium** | Captured token is unusable without proof key | Mitigation 1: Hook the DPoP key generation + nonce |
| | | | Mitigation 2: Use service principal instead (no DPoP) |
| | | | Mitigation 3: Inject the script that intercepts DPoP proof JWTs |

DPoP adds per-request signing. If the Dataverse API requires DPoP for the mobile app's client:

```javascript
// hook-dpop.js
Java.perform(function() {
    // Hook DPoP header injection — find where Authorization header is set
    var RequestBuilder = Java.use('okhttp3.Request$Builder');
    RequestBuilder.header.overload('java.lang.String', 'java.lang.String').implementation = function(name, value) {
        if (name.toLowerCase() === 'authorization') {
            console.log('[AUTH] ' + name + ': ' + value.substring(0, 60) + '...');
        }
        if (name.toLowerCase() === 'dpop') {
            console.log('[DPOP] DPoP proof JWT: ' + value);
        }
        return this.header(name, value);
    };
});
```

**Note:** Service Principal tokens are NOT DPoP-bound — yet another reason to prefer that approach for the canteen connector.

### 8.3 App Sandboxing & Anti-Tamper

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Frida detection | Medium | App force-closes | Use Frida Gadget (hidden) + MagiskHide |
| Root detection | Medium | App refuses to run | MagiskHide / Zygisk + Shamiko |
| Emulator detection | N/A (physical device) | — | We have real hardware |
| SSL pinning detection | Low | Mitm fails silently | Pre-test with a controlled endpoint |
| Obfuscation | **Certain** | Harder static analysis | Accept — focus on runtime hooking |
| App integrity check | Low-Medium | Detects modified APK | Use original APK; only hook at runtime via Frida |
| Certificate transparency | Medium | With CA is insufficient | Frida bypass on CT enforcement |

**Anti-detection checklist:**
- [ ] Magisk v30.7: Zygisk enabled, MagiskHide for target app
- [ ] Shamiko module: Hides Magisk from target app
- [ ] Frida: Use `--reuse` flag (don't start new process, attach to running)
- [ ] Frida: Add hook to disable `Debug.isDebuggerConnected()`
- [ ] Frida: Add hook to disable `android.os.Build.TAGS.contains("test-keys")`

```javascript
// anti-detect.js
Java.perform(function() {
    // Bypass Frida detection
    Java.use('android.os.Process')['sendSignal'].implementation = function(pid, sig) {
        // Block SIGTRAP used by some anti-Frida
        if (sig === 5) return;
        return this.sendSignal(pid, sig);
    };

    // Bypass root detection via Build.TAGS
    Java.use('android.os.Build').TAGS.value = 'release-keys';

    // Bypass debugger detection
    Java.use('android.os.Debug').isDebuggerConnected.implementation = function() { return false; };
});
```

### 8.4 Rate Limiting / API Governance

| Risk | Mitigation |
|------|-----------|
| 429 Too Many Requests | Add exponential backoff, respect `Retry-After` header |
| Service Protection API limits | 6000 requests per 5 min per user (Dataverse default) |
| Bulk extraction throttling | Split into batches, add delays between entities |

```python
# Rate limited curl wrapper
import time, requests, json

def call_dataverse(url, token, max_retries=5):
    headers = {
        'Authorization': f'Bearer {token}',
        'Accept': 'application/json',
        'OData-MaxVersion': '4.0',
    }
    for attempt in range(max_retries):
        resp = requests.get(url, headers=headers)
        if resp.status_code == 429:
            retry_after = int(resp.headers.get('Retry-After', 30))
            print(f"Rate limited — waiting {retry_after}s")
            time.sleep(retry_after)
            continue
        if resp.status_code == 401:
            print("Token expired — refresh needed")
            return None
        resp.raise_for_status()
        return resp.json()
    raise Exception("Max retries exceeded")
```

### 8.5 Legal & Compliance

- **Read-only mandate:** No modifications to production data
- **Data privacy:** Field Service data may contain PII (customer names, addresses, phone numbers). Handle with appropriate controls.
- **API ToS:** Dynamics 365 API usage subject to Microsoft Terms — bulk export may require Data Export Service or Synapse Link instead.
- **Token handling:** Never commit tokens to git. Use environment variables or a secrets manager.

---

## 9. Task Dependencies & Execution Order

### Dependency Graph

```
                    ┌───────────────────────────────────────┐
                    │   T0: Environment Setup                │
                    │   - ADB, Frida, mitmproxy installed    │
                    │   - Device rooted, Magisk configured   │
                    └────────────────┬──────────────────────┘
                                     │
                    ┌────────────────▼──────────────────────┐
                    │   T1: APK Acquisition & Decompilation │
                    │   - base.apk pulled from device       │
                    │   - jadx decompiled                   │
                    │   - Static strings extracted          │
                    └────────────────┬──────────────────────┘
                                     │
            ┌────────────────────────┼────────────────────────┐
            ▼                        ▼                         ▼
   ┌──────────────────┐   ┌─────────────────────┐   ┌─────────────────────┐
   │ T2a: Auth Flow   │   │ T2b: Frida Hooks    │   │ T2c: Traffic Setup  │
   │ - Identify MSAL  │──▶│ - Network intercept  │──▶│ - mitmproxy CA     │
   │ - Find client_id │   │ - Token capture      │   │ - Proxy configured │
   │ - Tenant discovery│  │ - Cert unpin         │   │ - Transparent mode │
   └────────┬─────────┘   └──────────┬──────────┘   └──────────┬──────────┘
            │                        │                          │
            └────────────────────────┼──────────────────────────┘
                                     ▼
                    ┌──────────────────────────────────────┐
                    │   T3: Interactive Session            │
                    │   - Login to Field Service app       │
                    │   - Navigate key screens             │
                    │   - Capture API calls + tokens       │
                    └────────────────┬─────────────────────┘
                                     │
                    ┌────────────────▼──────────────────────┐
                    │   T4: Data Catalog Generation         │
                    │   - Parse $metadata                   │
                    │   - Create entity list                │
                    │   - Document fields for canteen       │
                    └────────────────┬──────────────────────┘
                                     │
                    ┌────────────────▼──────────────────────┐
                    │   T5: Service Principal Setup         │
                    │   - Register app in Entra ID          │
                    │   - Grant permissions                 │
                    │   - Verify SP access                  │
                    │   - Token rotation strategy            │
                    └────────────────┬──────────────────────┘
                                     │
                    ┌────────────────▼──────────────────────┐
                    │   T6: Deliverable Package             │
                    │   - API endpoint map                  │
                    │   - Data catalog (JSON + Markdown)    │
                    │   - Auth config for canteen           │
                    │   - Sample API responses              │
                    └───────────────────────────────────────┘
```

### Execution Order

| Order | Task | Blocked By | Est. Duration | Parallel? |
|-------|------|-----------|---------------|-----------|
| T0 | Environment Setup | — | 30 min | ✅ First step |
| T1 | APK Decompilation | T0 | 1 hr | — |
| T2a | Auth Analysis (Static) | T1 | 45 min | ✅ T2a/b/c parallel |
| T2b | Frida Hooks | T1 | 1 hr | ✅ T2a/b/c parallel |
| T2c | Traffic Setup | T0 | 30 min | ✅ T2a/b/c parallel |
| T3 | Interactive Session | T2a + T2b + T2c | 2 hr | — |
| T4 | Data Catalog | T3 | 2 hr | — |
| T5 | Service Principal | T3 (for API surface knowledge) | 1 hr | ✅ Can run with T4 |
| T6 | Deliverable Package | T4 + T5 | 1 hr | — |

**Total estimated time:** 8-10 hours of focused work.

### Parallelization Notes

- **T2a, T2b, T2c** are fully parallel — different tools, no shared state
- **T4 and T5** can run in parallel after T3 completes
- The **Frida hooks** should be written and tested on a simple test app first

---

## 10. Tools & Versions

| Tool | Version (recommended) | Purpose |
|------|----------------------|---------|
| `adb` | platform-tools 35+ | Device communication |
| `frida` | 16.6.6 | Dynamic instrumentation |
| `frida-tools` | 12.5+ | Frida CLI |
| `jadx` | 1.5+ | DEX → Java decompiler |
| `apktool` | 2.10+ | Android resource extraction |
| `dex2jar` | 2.4+ | DEX → JAR conversion |
| `apk-mitm` | 1.3+ | Auto cert-pinning patch |
| `mitmproxy` | 10+ | TLS interception |
| `apkleaks` | latest | Secret scanning in APKs |
| `sqlite3` | 3.45+ | Local DB inspection |
| `python3` | 3.12+ | Scripting, data processing |
| `jq` | 1.7+ | JSON processing |
| `curl` | 8+ | API testing |

---

## Appendix A: Device Quick Reference

```bash
# ADB commands
adb devices -l                    # List devices
adb shell "su -c whoami"          # Confirm root
adb shell setprop service.adb.tcp.port 5555  # Switch to TCP mode
adb connect <device-ip>:5555      # Wireless ADB (for convenience)

# Frida quick reference
frida-ps -U                       # List processes
frida-ps -Uai                     # List installed apps
frida -U -f <pkg> -l script.js    # Spawn + inject
frida -U <pkg> -l script.js       # Attach to running

# Proxy toggling
adb shell settings put global http_proxy <ip>:8080
adb shell settings put global global_http_proxy_host <ip>
adb shell settings put global global_http_proxy_port 8080
# Clear proxy
adb shell settings put global http_proxy :0
adb shell settings delete global global_http_proxy_host
adb shell settings delete global global_http_proxy_port

# App info
adb shell dumpsys package <pkg> | grep -E 'version|flags|signatures'
```

## Appendix B: Common Dynamics 365 / Dataverse URLs

```bash
# Organization URL patterns
https://<org>.crm.dynamics.com/                    # North America
https://<org>.crm<x>.dynamics.com/                  # x = region number
https://<org>.crm.dynamics.com/api/data/v9.2/      # WebAPI base
https://<org>.crm.dynamics.com/api/fieldservice/   # FSO specific
https://<org>.crm.dynamics.com/main.aspx           # Web UI

# Discover your org URL from the device:
# 1. Check app cache/prefs
adb shell "su -c grep -r 'dynamics' /data/data/<package>/shared_prefs/"
# 2. From a network request in the traffic capture
```

## Appendix C: Frida Script Directory Structure

```
frida-scripts/
├── hook-okhttp.js          # Network call logging
├── hook-msal.js            # Auth token capture
├── hook-dataverse.js        # Response body capture
├── hook-sqlite.js           # Local DB query logging
├── hook-dpop.js             # DPoP interception
├── hook-cert-unpin.js       # Universal cert unpin
├── hook-anti-detect.js      # Anti-tamper bypass
├── token-collector.js       # Combined token capture
└── run-all.sh               # Launch scripts in order
```