import { timeStamp } from 'console'; import * as iink from 'iink-js'; import { action, observable } from 'mobx'; import * as React from 'react'; import { Doc } from '../../fields/Doc'; import { InkData, InkField } from "../../fields/InkField"; import { Cast, NumCast, StrCast } from '../../fields/Types'; import { DocumentType } from "../documents/DocumentTypes"; import './InkTranscription.scss'; export class InkTranscription extends React.Component { static Instance: InkTranscription; @observable _mathRegister: any; @observable _mathRef: any; @observable _textRegister: any; @observable _textRef: any; private currGroup?: Doc; constructor(props: Readonly<{}>) { super(props); InkTranscription.Instance = this; } componentWillUnmount() { this._mathRef.removeEventListener('exported', (e: any) => this.exportInk(e, this._mathRef)); this._textRef.removeEventListener('exported', (e: any) => this.exportInk(e, this._textRef)); } @action setMathRef = (r: any) => { if (!this._mathRegister) { this._mathRegister = r ? iink.register(r, { recognitionParams: { type: 'MATH', protocol: 'WEBSOCKET', server: { host: 'cloud.myscript.com', applicationKey: '7277ec34-0c2e-4ee1-9757-ccb657e3f89f', hmacKey: 'f5cb18f2-1f95-4ddb-96ac-3f7c888dffc1', websocket: { pingEnabled: false, autoReconnect: true } }, iink: { math: { mimeTypes: ['application/x-latex', 'application/vnd.myscript.jiix'] }, export: { jiix: { strokes: true } } } } }) : null; } r.addEventListener('exported', (e: any) => this.exportInk(e, this._mathRef)); return this._mathRef = r; } @action setTextRef = (r: any) => { if (!this._textRegister) { this._textRegister = r ? iink.register(r, { recognitionParams: { type: 'TEXT', protocol: 'WEBSOCKET', server: { host: 'cloud.myscript.com', applicationKey: '7277ec34-0c2e-4ee1-9757-ccb657e3f89f', hmacKey: 'f5cb18f2-1f95-4ddb-96ac-3f7c888dffc1', websocket: { pingEnabled: false, autoReconnect: true } }, iink: { text: { mimeTypes: ['text/plain'] }, export: { jiix: { strokes: true } } } } }) : null; } r.addEventListener('exported', (e: any) => this.exportInk(e, this._textRef)); return this._textRef = r; } transcribeInk = (groupDoc: Doc | undefined, inkDocs: Doc[], math: boolean) => { if (!groupDoc) return; const validInks = inkDocs.filter(s => s.type === DocumentType.INK); const strokes: InkData[] = []; validInks.filter(i => Cast(i.data, InkField)).forEach(i => { // TODO: interpolate missing times stamps const d = Cast(i.data, InkField, null); const left = Math.min(...d?.inkData.map(pd => pd.X) ?? [0]); const top = Math.min(...d?.inkData.map(pd => pd.Y) ?? [0]); strokes.push(d.inkData.map(pd => ({ X: pd.X + NumCast(i.x) - left, Y: pd.Y + NumCast(i.y) - top, time: pd.time }))); }); this.currGroup = groupDoc; const pointerData = { "events": strokes.map(stroke => this.inkJSON(stroke)) }; // console.log(JSON.stringify(pointerData)); const processGestures = false; if (math) { this._mathRef.editor.pointerEvents(pointerData, processGestures); } else { this._textRef.editor.pointerEvents(pointerData, processGestures); } } inkJSON = (stroke: InkData) => { return { "pointerType": "PEN", "pointerId": 1, "x": stroke.map(point => point.X), "y": stroke.map(point => point.Y), "t": stroke.map(point => point.time), "p": new Array(stroke.length).fill(1.0) }; } exportInk = (e: any, ref: any) => { const exports = e.detail.exports; if (exports) { if (exports['application/x-latex']) { const latex = exports['application/x-latex']; console.log(latex); if (this.currGroup) { this.currGroup.text = latex; this.currGroup.title = latex; } ref.editor.clear(); } else if (exports['text/plain']) { const text = exports['text/plain']; console.log(text); if (this.currGroup) { this.currGroup.text = text; this.currGroup.title = text; } ref.editor.clear(); } } } render() { return (
) } }