@pithagoras/office0.2.0

Office for Pithagoras

@pithagoras/office lets you and the agent work on the same Word (.docx) and Excel (.xlsx) documents. You edit them in the real OnlyOffice Document Server editor, embedded in the portal, which the extension runs in a container through Docker for extensions; the agent reads and edits the same documents through tools of its own, in an editor of its own beside you, so you see its changes as it makes them. An Office tab holds the library.

This extension is not made or endorsed by Ascensio System SIA. "ONLYOFFICE" names what it runs.

What it needs

• Pithagoras 0.3.0 or later. For agents to answer comments and the editor's chat, a portal whose extension API has chats.start, chats.lastAnswer, chats.owner and http.person (thecodacus/pithagoras#339); on an older one the rest of Office works, and the Office page, the editor's bar and the agents' office tools say that agents do not answer there. Such a portal cannot say whose a chat is, so there a chat's panel lists documents, and its agent has office tools, only where the chat works in an agent's home or a project's folder itself, not in a folder inside one.
• Docker for extensions (pithagoras-docker 1.3.0 or later). It ships with the portal but is not on by default; installing this extension from npm brings it along.
• A Linux host with Docker Engine 28.0 or later that the portal can reach (its socket): the portal on the machine itself, or in Docker (both of Pithagoras' Compose files). On macOS or Windows the portal has to run in Docker.
• Recommended: 1 GB of RAM. About 700 MiB were measured while one document is edited, and 770 MiB to 1.05 GB with three documents open in the agents' editors (the most there are at once: 540-975 MiB of it the agents' browser), 800-940 MiB with people's editors of two of them open as well. The settings page shows the memory all of Office uses while it runs: the Document Server's and, while it runs, the agents' browser's. That browser starts on demand, stops 10 minutes after its last editor closed, and is capped at 3 GB. The figure is what docker stats shows: the containers' memory less the file cache the kernel can drop again.
• Disk for the images (onlyoffice/documentserver, several GB, Node's node:22.23.4-alpine3.23, about 50 MB, for the relay below, and for the agents' editor chromedp/headless-shell:157.0.8103.15, Chromium's headless shell, about 540 MB), which the portal pulls on the first start, all at once, with the progress of each shown in Settings.
• Behind a reverse proxy (one that ends TLS, say), a browser that says which page a request comes from, as every current one does (Sec-Fetch-Site), or a proxy that passes X-Forwarded-Host and X-Forwarded-Proto (or keeps Host): the Document Server names the addresses it hands the editor (the opened document among them) by the address the browser used, and an http:// address on an https:// page is refused by the browser, which shows as "Download failed".

Installing

In Settings → Extensions → Discover → Install by name: npm:@pithagoras/office. Then open Settings → Office and press Start container (that switches Office on too). The image is pulled on the first start (onlyoffice/documentserver:10.0.0.1 by default, the full tag of the build this version was tested with, never :latest or a shorter tag like 10.0, which move to each new build: a new build can change the editor's callback and token behaviour, so it is yours to change on purpose, in the same page). A new Document Server container answers within seconds but then makes its fonts for about two minutes and restarts its own services; until it is done, Settings says it is starting and the editor waits for it, as an editor opened before would lose what is typed in it. The same start downloads the agents' editor beside the Document Server. Settings → Office shows Office as one thing: stopped, downloading (the progress of every download under way), starting, running, or failed, with the error naming the part that failed (the Document Server or the agents' editor). Until the agents' editor is there the agents read the saved files and cannot edit; an install that had the Document Server before gets it with the next Start container, and the page says it is missing until then. It is pinned by its digest: a tag that came to name another build is refused, not run, and said as the error.

What you get

• The library, on the portal's Office tab: create blank Word documents (text in Arial 11, set in the document's own defaults, so a built-in table style shows one font in every row; a document you add keeps its own) and spreadsheets (Calibri 11, with the workbook's styles part, which the editor needs for a table over a bold or linked cell; a workbook you add keeps its own), open one full screen in the embedded editor, sort the list by name, modified, size or type, and download a copy. Every document lives in an agent's home or a project's folder and belongs to that agent or project; there are no documents of nobody's. Every .docx and .xlsx in the agents' homes and the projects' folders (not the folders other chats work in, nor the rest of the workspace) is indexed when the extension starts, and again with Rescan, in place: the file is the document, and editing it edits that file. New document on the Office tab makes it in the home of the agent New starts your chats with (the first agent's, unless you picked another on its Agent page); the one in a chat's panel, and the agent's office_create, make it in that chat's agent home or project folder, as the portal says whose the chat is. One whose file is gone (a git checkout of another branch, say) stays listed for a week, marked, with its owner and grants, and is the same document again when the file comes back. Deleting one moves its real file into the library's trash (.trash/<time>-<id>/ in the data folder), with document.json, which says where it was, and what was kept for it; the entry is kept for 7 days and then removed. There is no button to restore it, but the file can be moved back by hand.

• A panel beside each chat with the documents that chat's agent or project sees (below). A document opened there opens in the editor right in the panel, beside the conversation, so you watch the agent's edits while you talk to it: ← All documents goes back to the list, and a small button opens it in the Office tab instead. The panel can be docked left, right or at the bottom and dragged as narrow or wide as you like; the editor fills it. A card in the chat for each call of the agent's tools that names its document opens that document in the same panel (a call that lists the documents opens the list). The same document open in the panel and in the Office tab (or in two browser tabs) is one editing session seen twice.

• The agent's tools, in the chats of agents and projects (a chat in a folder inside a project is the project's, as the portal says whose a chat is; the panel beside it lists by the same rule): office_list, office_create, office_read, office_edit, office_format, office_comments and office_screenshot. An agent sees the documents in its own folder (its home, or its project's) and the ones it was granted by a mention; another agent's, and the files in another agent's or project's folder, are not there for it, and asking for one gets the same answer as for a document that does not exist. What an agent makes goes into its folder and belongs to it, not to one of its chats. A file in its folder is read through the portal as the agent itself would read it: with the sandbox on, one the agent could not read is refused, and a link is not followed. A document in another agent's home or project folder that you opened to it by a mention is read with the portal's rights (your mention says it may), and changed only where the agent could write that folder itself: with the sandbox on, it reads and answers in comments.

• The agent edits beside you, live: each agent (each agent's or project's folder) that works in a document has an OnlyOffice editor of its own open in it, in a browser of Office's own (the agents' editor), in the same editing session as the people in the document. People see it there as " (agent)", with its cursor. office_read gives the document as it stands in that editor, with what people typed and have not saved yet; office_edit makes its edits there, so they show in your editor at once, and nothing you typed is lost or written over. An edit is made whole or not at all: every quote, sheet name and position is checked against the document as it stands first, and an edit that fails after it changed something is taken back. Where even that fails (a person changed the same sheets meanwhile), the agent hears so (error:partial) and reads the document again. In a spreadsheet, what you changed yourself after the agent's step (a cell you typed into or formatted, a row or a sheet of the call's you wrote into, frozen panes you moved) is never taken back with the agent's call: it stays as you made it, and the agent's answer (error:partial) names it. After an edit the Document Server is asked to save the session, and the agent hears the new revision; that save holds what you typed until then too. Word: replace quoted text, add paragraphs (in any paragraph style the document has) or a table after a paragraph, remove a paragraph (a document keeps one: its only paragraph removed leaves it empty). A quote is looked for in the body's text only: not in headers, footers, footnotes or a table of contents' entries (which repeat the headings), and a text found more than once is refused unless the call says which place it means (nth, from 1, in reading order). A list item's number or bullet is no part of its text: office_read gives the list in the brackets ([5] (bullet) Point). How a Word document looks is office_format's, a tool of its own (so a chat that never formats can leave it lazy in Settings → Tools, and pay nothing for it): characters (font, size, bold, italic, underline, strike, superscript, subscript, colour, highlight, links on and off, character styles, and taking a text's own formatting off), paragraphs (alignment, spacing, line spacing, indents, bulleted and numbered lists with their levels and a new start, page break before, keep with next), styles (new paragraph and character styles; an existing one changes only when the call says change: true, and the answer counts the paragraphs that use it), tables (a table style, width, column widths, header rows, borders, cell fills, alignment, row heights, merged cells), the page (size, orientation, margins, headers and footers with the page number and count, a first page of its own), sections, page breaks, footnotes and a table of contents. A table of contents follows the document: every office_edit and office_format call that changes a document holding one brings its entries and page numbers up to date as the last part of the same step (so a failed call's Undo takes that back too), and its answer says so; what was changed by hand in the entries is overwritten then, as Word's own update does. A comment on an entry is made again on the text of the new entry (the one with its text, else the one at its place), with its author, time, state and replies, so its thread and its chat go on; the answer says that too. A table of contents that lists nothing yet is found by the editor's English text for it ("No table of contents entries found.", as Office's editors and an English Word write it): one made in a Word of another language is not brought up to date until it lists something. With two tables of contents that list nothing yet in sections of different widths, the second one's new entries take their page-number tab at the first one's width. Each update lays the whole document out: in the agent's editor about 17 seconds for 340 pages of text (a file of 130 kB), so an office_edit or office_format call has more time the larger the document's file is (two minutes at most). Each op names its place by a quote, as office_edit's do. A call is made whole or not at all: everything it names is checked first (quotes, styles, tables and cells, what a header holds), and a call that fails after it changed something is taken back by the editor's own Undo, as one step. A header or footer that holds more than text (a picture, a table, a field other than the page number and count) is replaced only with replace: true; setting the header of a later section that showed the one before it gives it one of its own, and the earlier one stays ("" there gives it an empty one, so its pages show none). A link is changed or taken off whole: a quote of part of a link is refused. A style is never based on itself, also not through others. A section break's type (continuous, evenPage, oddPage) is how the section after it starts, and a page size alone keeps the section's orientation. Lengths are points, or a text with its unit ("2.5cm", "10mm", "1in"), colours "#RRGGBB". Borders are single lines (the editor draws no other kind through its API). When a call moves the pages (page setup, a style changed, a section, a page break, a table of contents), its answer says so, and office_screenshot shows them. office_read with formatting: true also says how a Word document looks: the document's defaults and the styles in use (each as it is itself; what it inherits from another is not added), each section's page, margins, header and footer (or the earlier section's it shows), the footnotes and where each one's mark is (a table's in its cell), and each paragraph's and run's own formatting (not what its style gives it; what it sets off or to zero is said too, "not bold", "indent left 0cm", as it hides the style's), a numbered item's number as the editor gives it ((numbered "3."), so a list that goes on and one that starts again are told apart), a table's style, header rows, merged cells and borders, and each cell's own fill, vertical alignment and borders with its paragraphs' and runs' own formatting (B2: fill #FFFF00, valign center; (right) "42": bold, under the table's rows; a table or a content control inside a cell is the cell's text, its paragraphs counted and read with the cell's); 50,000 characters of it at most. Not read: the saved file's formatting, when Office is stopped. Excel: write values and formulas into a range of any sheet (the editor computes the formulas; =Data!A1 and ='My data'!A1 reach other sheets), insert and delete rows and columns ({insertRows: "5:6"} makes new rows 5 and 6; formulas follow the cells they name, and those that named deleted cells turn #REF!), and make, rename, move, copy, hide, show and delete sheets, up to 20 of these in one call (Excel's rules for names: at most 31 characters, none of []:*?/\, no two alike in any case; one sheet always stays visible). A hidden sheet is moved by showing it, moving it and hiding it again. Deleting a sheet turns the formulas that name it into #REF!, as in Excel. office_read lists every sheet with its position and whether it is hidden, and reads one sheet or all of them. A change to the sheets shows in your editor once the Document Server has let it through, a moment after the call makes it. A sheet the edit deletes is hidden and renamed ~deleted … at once, and goes when the edit's other ops are made. Deleted rows and columns cannot be taken back, so a call deletes them last, after every op that can be (a call that does otherwise is refused); where the call fails after a delete was made, what it changed below or right of the deleted lines stays too (its cells moved), and the answer names those ops. A call inserts and deletes 20,000 rows and columns at most. Rows and columns are inserted and deleted while you are in the workbook too: the agent's answer then names the people in it, as their cells move. If you are in the middle of typing into a cell when the agent's insert moves it, OnlyOffice 10.0 puts what you type at the cell's old address (a row off after a row inserted above it), and your own editor may show the cell empty until you reload it: the Document Server's co-editing, reported as ONLYOFFICE/DocumentServer#3867. Your editor's view can also differ when the agent writes the very cell you are typing in. How a spreadsheet looks is office_format's too: cells ("A1:D9", or whole columns "C:D" and rows "5:6") take a number format ("#,##0.00", "0%", "dd.mm.yyyy"), font, size, bold, italic, underline, strike, superscript and subscript (in cells that hold text: OnlyOffice sets them on a text's characters), colour, fill, alignment, wrap, text rotation, borders (single, double, dashed or dotted lines), merging and splitting, and taking all of a cell's formatting off (clear, merges too, and the conditional formats of those cells: OnlyOffice takes them off with it, and a call taken back cannot make them again, so the answer names them; the values stay). Columns take a width (in characters, as the sheet shows it), rows a height (in points), and both can be hidden, shown and fitted to their contents; frozen panes keep the rows above and the columns left of a cell in view. A call is made whole or not at all: it is checked against the workbook first (superscript on a number, a freeze of a hidden sheet, more than 20,000 cells, whole columns and rows counting the used cells in them, are refused before anything changes), each op reads what it changes first, and where a later op of the call fails, what it changed is put back. Some things cannot be put back, because OnlyOffice's API cannot read them: the alignment and the borders, the formatting of a text whose characters differ (part of it bold, say) and the width of a hidden column. Where a later op fails, they stay as the call set them, and the answer (error:partial) names them. OnlyOffice's API takes no border off and sets no "general" alignment back: only clear takes them off, with everything else. A merge keeps the top-left cell's value (as in Excel); taking a merge back puts the others' values back. A column width or row height put back is the column's or row's own from then on (the API sets none back to the sheet's default), which looks the same. Besides the cells' looks, office_format makes conditional formats (a rule on cells, or a whole column: a comparison with a value, a formula, a colour scale, data bars, the top or bottom n, duplicate or unique values; with a fill, a font colour, bold, italic, underline or strike where the kind has a look), data validation (a list of entries or the cells that hold them, whole numbers, decimals, dates and text lengths with a condition such as ">= 0" or "between 1 and 10"; or taking it off; not an error message, which OnlyOffice's API sets in the agent's own editor only and sends nowhere), links in cells (http, https or mailto; a cell that holds a text keeps it, one that holds a number or a formula needs the text to show, as OnlyOffice would put the address there), charts of a range with its header row and label column (column, stacked column, bar, stacked bar, line, pie, doughnut, area and scatter; title, axis titles, legend, place and size), a filter on a range with one column's criteria (or taking the sheet's off) and a formatted table of a range, in the editor's default style (the API has no other). office_edit sorts the rows of a range by up to three of its columns (the cells' formats and formulas move with their rows; merged cells are refused) and makes, changes and deletes named ranges. The same rule holds for them as for formats: a call is checked first, and where a later op fails what the call made is put back: a rule the call added is deleted (one a person changed since stays, and is named), validation goes back to what the cells had (cell by cell; the messages of a person's validation, and whether it ignores blanks, shows a dropdown in the cell, an error alert and an input message, are lost with the validation the call replaces, and the answer says so; OnlyOffice 10.0's editor takes no validation message from the API, ONLYOFFICE/DocumentServer#3869), a link to the text, look and link the cell had (the link's look replaces the cell's font, so a bold cell with a link comes back plain, and the answer names its font; a link a person changed meanwhile keeps its cell as they made it), a sort to the values, formulas and readable formats the range had (the alignment and borders it moved cannot be read back, so the answer names them), a name to its old reference or none. OnlyOffice's API takes no chart, no table away again, and a filter only whole (filter: "none" takes the sheet's off), and a rule cleared cannot be made again: these go last in a call, and the answer names the ones made before the op that failed, which stay. Whether a table overlaps another one is not known to the API, which ignores it without a word, and a table over a cell with a link is refused (in a workbook without a styles part, xl/styles.xml, OnlyOffice 10.0 can no longer read the formats of a linked or bold cell a table covers, ONLYOFFICE/DocumentServer#3870; the workbooks Office makes have the part and do not break, but the editor's API cannot tell the two apart, so the refusal stays for all of them, and office_read leaves out a cell whose format it cannot read, and says so). A sort moves the links of its cells with them, and where a later op fails the links stay where the sort put them (the answer names them). A sort re-orders the rows of its range: while you are in the workbook the agent's answer names the people in it, as it does after inserted or deleted rows (merging cells moves none, so it does not). A text in a rule or a list of a validation cannot hold a double quote (the file would hold a broken formula), a sheet with more than 2,000 conditional formats takes no new one (each is made by reading all of them). A cell under a conditional format reads the rule's fill as its own in office_read (the API gives what is drawn); taking back a format of such a cell can then set that fill on it. office_read with formatting: true says how each sheet looks: the font most cells have, the cells that look otherwise (grouped into ranges), merged cells, the columns' widths and rows' heights that differ from the sheet's, hidden ones and frozen panes, its conditional formats, the validation of the cells it lists (of the used range, 20,000 cells at most), its links, its filter and its charts, 50,000 characters of it at most. It cannot say the alignment or the borders (OnlyOffice has no getter for them), a formatted table (its style, its total row), the data a chart shows or the workbook's named ranges (none is listed by the API; the agent deletes the ones it made by name): office_screenshot shows the first three. Pictures are office_edit's too: the agent puts a PNG, JPEG or GIF from its own folder into a Word document ({after?, picture, width?, height?, alt?}: inline in a paragraph of its own after the paragraph that holds after, or at the end) or a spreadsheet ({sheet?, picture, at: "B2", width?, height?, alt?}: its top left corner in that cell). picture is a path below the agent's home or the project's folder, never an address; the file is opened through the portal as the agent itself would read it (with the sandbox on, one it could not read is refused, and a link that leads out of the folder is not followed), told by its first bytes and not by its name, and checked by its header before anything goes to the editor: 5 MB a picture, 10 MB and 20 pictures a call (MB of 1,000,000 bytes; a file used twice counts twice), 40 megapixels (a small PNG can unpack to gigabytes; a GIF's frames count where they reach past its screen) and 100 frames of a GIF (the editor shows its first). The picture goes to the editor as it is, up to its end marker (bytes after it are left out), so the saved file holds the file's own bytes, its metadata too: a photo's EXIF (where it was taken, the camera) goes into the document with it. Its data is outside the ops' own 1 MB limit. Its size is its own at 96 dpi (a JPEG turned by its EXIF orientation as it is shown), made smaller to fit the page's text area (Word) or 16 × 12 cm (a sheet), or the width and height given (a length like "6cm", a number is points, 0.2 to 50 cm; one given keeps the proportions, and the other side may not come out past 50 cm); its alt text is the file's name unless alt is given ("" gives none). A call that fails after its pictures is taken back like the other ops: in Word by the editor's Undo, in a sheet by deleting each picture the call made, except one you moved, resized or described meanwhile, which stays as you made it and which the answer names. (OnlyOffice's builder API has no way to delete a sheet's drawing; Office does it with the plugin method RemoveOleObject, which in 10.0.0.1 deletes any drawing of a sheet by its id.) office_read lists the pictures, with or without formatting: under a Word paragraph with their size and alt text (under a table by their cell, A2: a picture …, for one put after a cell's text), per sheet after its cells (Pictures: B2 4 × 2 cm "Logo", 50 at most); several in one place carry #1, #2. The pictures a document has, a person's and the agent's own, are changed by {editPicture, …} ops of office_edit: deleted, moved, sized or given new alt text. No crop, rotation or wrapping.

  Naming a picture. The agent names a picture by what office_read lists it under, and the two sit side by side:

  office_read says	the op that changes that picture
  [2] Their text and under it a picture 4 × 2 cm, alt text "Their logo"	{editPicture: 2, width: "8cm"}, {editPicture: 2, delete: true}
  [4] table, under it B2: #1 a picture 2 × 1 cm, … and B2: #2 a picture 1 × 1 cm, …	{editPicture: 4, cell: "B2", n: 2, alt: "New words"}
  [3] Floating text and under it a picture 3 × 3 cm, alt text "Their floating"	{editPicture: 3, move: true, after: "Closing paragraph"} (to the end: {editPicture: 3, move: true})
  Pictures: B2 7 × 4.2 cm "Wider red"; B8 #1 1.6 × 0.8 cm "First blue"; B8 #2 2.6 × 1.3 cm "Second green".	{sheet?, editPicture: "B8", n: 2, at: "B14"}, {editPicture: "B2", height: "3cm"}, {editPicture: "B8", delete: true}

  In Word editPicture is the number in brackets that office_read gives the paragraph or the table (a content control's paragraphs are counted in reading order), cell is the table cell the picture is listed under, and n (from 1) says which of several listed there: they carry #1, #2 in the listing, and only where there are several. In a sheet editPicture is the cell the picture is anchored at (its top left corner), and n which of the pictures anchored there. The numbers are those office_read showed before the call, in Word and in a sheet: every picture the ops name is found before the first op changes anything, so a call's own inserts, deletes and moves do not renumber them (two deletes at one cell are n: 1 and n: 2, and a picture moved to a cell is not the first there). An op that names a picture an earlier op of the call has taken away (deleted or moved) is refused, as is a picture that is not listed before the call (one the same call puts in cannot be edited by it, except on a sheet the call makes itself, where the op looks at its turn). The answer repeats each changed picture with what it was (Pictures: deleted the picture [2], which was 4 × 2 cm, alt text "Their logo"), so that a number that meant another picture (a person changed the document since it was read) shows; a sheet picture a person deleted since the call began is told so, and is not swapped for another.

  What an op does. An op does one thing:

  • delete: true.
  • A move. Word: move: true and after, nth, as an insert places a picture, in an inline paragraph of its own after the paragraph that holds after, or at the end. A sheet: at, the cell its top left corner moves to. A model that wrote the destination under the other of the two, or under to, is understood, and keys given as null and delete or move as false are left out, as the model of the real-portal tests filled in every key of the schema.
  • A new width, height and alt (a length like "6cm", 0.2 to 50 cm). One side keeps the picture's proportions as it is now, and the other may not come out past 50 cm. OnlyOffice cannot take a picture's alt text away, so alt is a text.

  A deleted or moved picture takes its paragraph with it where that held nothing else. It stays where it is the only paragraph of a table cell or the document, where it ends a section, and where removing it would leave a table last in its cell or the document or join two tables (Word draws those as one table); a picture inside a paragraph with text leaves the text. A moved picture is the same file (a copy of its media: not made again as a PNG), inline from then on, with its alt text and title. A cropped, rotated or wrapped picture is changed in its size and place only, and the text inside a picture is not read (the guard does not see it either).

  Undone, and a person's change. All or nothing as for the other ops:

  • Word: by the editor's Undo, which takes back the whole call. A person's change to the same picture between the agent's read and its call is not guarded (the answer names what the picture was), and what the Undo does to a change a person made to that picture between the call's first op and its failure was not measured: Word is not covered there.
  • A sheet: a move, a size and an alt text are put back by the steps that made them, except where a person changed the picture since (theirs stays and the answer names it), or where rows or columns were deleted for good after the move (then it stays where the call put it and the answer names the op), or an alt text where the picture had none (the editor cannot put none back).
  • A deleted sheet picture cannot be taken back: OnlyOffice reads a picture's bytes back only as a PNG it draws of the picture (a JPEG or GIF would not be the file it was), so a delete goes last in a call, as deleted rows do, and where a later op of the call fails (a delete of another picture, say) the answer names what stays deleted.
  • The delete is done by the plugin method RemoveOleObject, as the inverse of a picture put in is. It reaches the sheet the editor shows, so the agent's editor shows the picture's sheet for a moment and the sheet it showed before again; a hidden sheet's picture is not deleted (the op is refused: show the sheet first, in the call before the delete or in another).

  The agent cannot write comments or tracked changes (it reads comments with office_comments). Calls on one document run one at a time. A document nobody has open is opened by the agent's editor alone, and saved when it closes. While an agent's editor is in a document its session stays open after the people left it, so it is saved as the last person leaves: what they typed is in the file then, not only once the agent's editor closes. The agent's editor of a document closes after five minutes without a call; at most three are open at once (the least recently used one closes for a fourth); the browser stops ten minutes after its last editor closed, and starts again with the next call (a few seconds). Where you stopped Office, office_read reads the saved file and says so, and office_edit is refused until it runs again: there is no other way to write. The agent sees what it made: office_screenshot gives it a picture (JPEG) of a document as its own editor shows it: a Word page (by number, or the page that holds a quote), laid out once the editor's page count holds still and drawn once the document's pictures have loaded (the editor shows a document before its pictures come, and draws nothing where one is missing: the picture waits up to 6 s for them, and where one has still not come, or could not be loaded, takes the page as it is and says so to the agent; the same for the pictures the agent has just put in); or a range of a sheet with its column letters and row numbers, at 100 % zoom, at 75 % where it does not fit, and refused beyond that with the largest range from its first cell that fits (about 23 columns by 41 rows of default cells at 100 %, 30 by 55 at 75 %); hidden rows and columns are left out. By default a sheet's used range from A1, as much as fits. 1,024 px on the longer side (large: as large as the editor draws it, up to 1,568 for a sheet range, about 1,170 for a Word page). The picture is drawn from the canvases the editor draws the page or the cells on (a sheet's with its charts and pictures), not from the one above them where it draws selections, cursors and the names of the people in the document: so it shows that document alone, with no other editor in it, and no person's selection, cursor or name (measured with a person selecting and typing in the same sheet, and with their cursor on the same page). The agent's own active cell shows only as its highlighted column letter and row number. The text the picture shows comes with it, as office_read gives it, so the guard reads it: a Word page's body text (of a table over many pages, the rows on the page), or the cells of the range that are shown; 20,000 characters at most. Only in the picture, so the guard does not read them: a page's header, footer and footnotes, text boxes, shapes and charts, text inside the document's own pictures, and a long text that runs into the range from a cell outside it. A model that cannot take pictures is refused (error:no_images) before anything is done. A picture costs the model about 1,000 to 1,600 input tokens on every later turn of the chat. Taking one moves the agent's own view (its page, zoom, sheet and selection), which you may see as its cursor moving; pictures are taken one at a time in the agents' browser, and a sheet a person hid, moved or renamed meanwhile is not shown (sheet_changed). The page is laid out as the editor lays it out: with its fonts (Carlito for Calibri, for instance), so a page break may fall elsewhere than in desktop Word. A save that would overwrite a newer revision (the file changed on disk while it was open: a script, a git pull) is not lost and does not overwrite it: it is kept beside the document as "