-
Sync Media Document
The SMIL document including all the Sync Media extensions.
-
Media Object
A media resource in the Sync Media Document referenced using a
src
attribute. -
Media Object Reference
A reference to a Media Object in the Sync Media Document. A Media Object Reference may override attributes and parameters defined on the Media Object.
-
Media Param
Named parameters that can be defined on a Media Object and Media Object Reference to communicate options to the Media Object Renderer
-
Media Object Renderer
A component used by the Sync Media Player to render Media Objects of one or more media type.
-
Sync Media Player
A user agent that knows how to process and play back Sync Media Documents.
-
Timeline
A timeline that lays out Time Containers in a linear mode.
-
Time Container
Container elements that dictate the playback mode for its child elements.
-
Parallel Time Container
A Time Container which child elements will all play in parallel.
-
Sequential Time Container
A Time Container which child elements will all play in sequence.
This specification describes a file format that lets an author orchestrate the playback of a wide variety of media types in a linear timeline. Sync Media files are rendered to a user by a Sync Media aware player.
Even though the Sync Media spec does support the implementation of generic Sync Media Players, it is reasonable to expect that most implementations will be specialized and refer to one or more of the media types in the Sync Media timeline as being "primary".
An example of such an implementation is an EPUB reading system that will consider the EPUB media type as primary, presenting the public ation as the main content and the other media types as complementary.
Another example would be a system focused on video playback. Such a system would consider any video format as primary and build its presentation of the Sync Media timeline focusing on this aspect of the playback.
Just as EPUB Media Overlays, Sync Media is based on a subset of SMIL 3.0 features (https://www.w3.org/TR/REC-smil/smil30.html). It is also designed to be "backwards compatible" with its sister spec Media Overlays in order for future synergies.
In comparison to "traditional" SMIL, Sync Media Documents are layout agnostic. The layout
and region
concepts found in SMIL are not present, instead these aspects are left to the Sync Media Player. Thus the Sync Media format and the Sync Media Player can be embedded in systems such as EPUB reading systems etc, where these systems can customize the presentation according to their respective needs.
It is possible that encodings other than XML, such as JSON, could be used to persist and distribute Sync Media Documents.
A Sync Media file can be "containarized" to allow the embedding of some or all of the resources that are referenced by its Sync Media Document. The container format used in these cases is the OCF Zip Container (https://www.w3.org/publishing/epub3/epub-ocf.html#sec-container-zip minus EPUB specific semantics). The OCF format has all of the features, such as encryption; signing etc, needed for the current version as well as foreseeable future versions of Sync Media.
One of the objectives of the Sync Media specification is to let a Sync Media Player add synchronized media, such as Media Overlays, to an existing resource without the need to alter the resouce. This will help the adoption of synchronized media features in the accessibility space by enabling "non-destructive" methods to adding additional modes of narration to existing media resources.
Even though many of the Modules present in SMIL are out of scope for the Sync Media specifications purposes, the SMIL 3.0 specification also has concepts that can be used without modification.
-
BasicMedia
- xml:id
- src
- type
-
MediaPanZoom (https://www.w3.org/TR/REC-smil/smil30.html#smil-extended-media-object-q31)
- panZoom
-
MediaClipping (https://www.w3.org/TR/REC-smil/smil30.html#smil-extended-media-object-smilMediaNS-MediaClipping)
- clipBegin
- clipEnd
-
MediaParam (https://www.w3.org/TR/REC-smil/smil30.html#smil-extended-media-object-smilMediaNS-MediaParam)
- name
- value
- valuetype
-
MediaAccessibility (https://www.w3.org/TR/REC-smil/smil30.html#smil-extended-media-object-smilMediaNS-MediaAccessibility)
- alt
- longdesc
- readindex
-
Metainformation Module (https://www.w3.org/TR/REC-smil/smil30.html#smil-metadata-smilMetadataNS-metaModule)
Note regarding Timing and Syncronisation
The Timing and Synchronization features in SMIL are very comprehensive. For the purposes of this specification the complete set of SMIL's timing features add too much complexity. A simplified extension module, SyncMediaTiming, is therefore added in order to offer more a more tailored solution.
SMIL already supports adding additional synonyms to its ref
element to make documents easier to read. It also supports the addition of extra Media Object Modules that can add attributes to the various existing modules.
Sync Media extends SMIL's Media Objects by adding additional modules with accompanying attributes / params to convey locator refinements, rendering instructions etc. All additional modules and attributes are prefixed with the Sync Media namespace (in the examples "sync:
").
Sync Media SyncMediaObject
s support Media Fragment selectors (https://www.w3.org/TR/media-frags/) where such are applicable to their media type.
-
SyncMediaObject (media)
- lang
- class
-
SyncMediaObjectReference (mediaref)
- idref
- selector
-
SyncMediaObjectText (text)
- writingDirection (equiv to https://www.w3.org/TR/SMIL3/smil-text.html#adef-textWritingMode)
-
SyncMediaObjectAudio (audio)
- volume
- pitch
- pan
-
SyncMediaObjectVideo (video)
- volume
- playbackrate
-
SyncMediaObjectEPUB (epub)
-
SyncMediaObjectPDF (pdf)
- page
- viewrect
- nameddest
NOTE: See https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdf_open_parameters.pdf
-
SyncMediaObjectHTML (html)
-
SyncMediaClipping
- clipPath
NOTES:
clipPath
allows you to specify a clipping path using a SVG path definition. The clipping is applied to the visible region of the Media Object on which it is defined. When combined with thepanZoom
module theclipPath
is applied inside the rect defined by thepanZoom
attribute. -
SyncMediaTiming
- begin
- end
- fill
NOTES:
begin
andend
attribute values can only be simple length units.fill
can have the following values, "repeat", "freeze", "reset" and "remove". -
SyncMediaUserInteraction
NOTES: Add specific attributes related to interactive playback modes for Media Objects
A Sync Media SMIL document can be composed of multiple "root sequences" using xlink
functionality. This allows for more effective handling of large volume documents as you can divide the in to more manageable fragments.
To limit complexity it is suggested that linking of SMIL documents is only allowed on direct seq
ascendants to the body
element.
<smil>
<body>
<seq xlink:href="part-1.smil" />
<seq xlink:href="part-2.smil" />
</body>
</smil>
In order to have time to get validation of the features described in this document from implementors, it is suggested that features are rolled out in phases.
-
Phase 1
This phase has feature parity with EPUB Media Overlays. In other words Basic SMIL Media Objects of type
text
andaudio
are supported. No additional timing functionality is implemented apart from what is afforded by the Time Containers. -
Phase 2
This phase adds support for the
SyncMediaObject
s. -
Pahse 3
This phase adds support for
SyncMediaTimeline
,PanZoom
andSyncMediaClipping
.
<smil>
<head>
<meta property="media:active-class">-epub-media-overlay-active</meta>
<meta property="media:playback-active-class">-epub-media-overlay-playing</meta>
</head>
<body>
<par id="cover" type="cover">
<audio src="../audio/narr_cover.mp4" clipBegin="0s" clipEnd="8s" />
<text src="cover.xhtml#title" />
</par>
<seq id="part_1" type="part">
<seq id="chapter_1" type="chapter" textref="chapter_1.xhtml">
<par>
<audio src="../audio/chapter_1.mp4" clipBegin="10s" clipEnd="14s" />
<text src="chapter_1.xhtml#f000001" />
</par>
<par>
<audio clipBegin="15s" clipEnd="23s" />
<text src="chapter_1.xhtml#f000002" />
</par>
</seq>
</seq>
</body>
</smil>
<smil>
<head>
<sync:media type="application/zip+epub" id="epub" src="https://ebooks.com/titles/theexcitingbookofwords/theexcitingbookofwords.epub">
<param name="active-class" value="-epub-media-overlay-active" valuetype="data" />
<param name="playback-active-class" value="-epub-media-overlay-playing" valuetype="data" />
</sync:media>
<sync:media type="audio/mp4" id="narr_cover" src="https://ebooks.com/titles/theexcitingbookofwords/narration_cover.mp4" />
<sync:media type="audio/mp4" id="narr_c1" src="https://ebooks.com/titles/theexcitingbookofwords/chapter_1.mp4" />
</head>
<body>
<par id="cover" type="cover">
<sync:mediaref idref="narr_cover" clipBegin="0s" clipEnd="8s" />
<sync:mediaref idref="epub" sync:selector="epubcfi(/6/4![cover]/4[0:67])" />
</par>
<seq id="part_1" type="part">
<seq id="chapter_1" type="chapter">
<par>
<sync:mediaref idref="narr_c1" clipBegin="10s" clipEnd="14s" />
<sync:mediaref idref="epub" sync:selector="epubcfi(/6/6![chapter1]/4[0:49])" />
</par>
<par>
<sync:mediaref idref="narr_c1" clipBegin="15s" clipEnd="23s" />
<sync:mediaref idref="epub" sync:selector="epubcfi(/6/6![chapter1]/4[50:144])" />
</par>
</seq>
</seq>
</body>
</smil>
<smil>
<head>
<sync:media type="application/lpf+zip" id="audiobook" src="https://audiobooks.com/titles/theexcitingbookofwords/theexcitingbookofwords.lpf" />
</head>
<body>
<par id="cover" type="cover">
<image src="https://audiobooks.com/titles/theexcitingbookofwords/cover.png"
alt="Book title in large exciting font. The background is filled with a multi-colored flurry of words set in varied fonts." />
<sync:mediaref idref="audiobook" sync:selector="t=0, 8" />
<text>
The exciting book of words by L Zamenhof. Read by Lorenzo Vallinio
</text>
</par>
<seq id="part_1" type="part">
<seq id="chapter_1" type="chapter">
<par id="chapter_1_par1">
<sync:mediaref id="chapter_1_par1_audio" idref="audio" sync:selector="t=10, 14" />
<text id="chapter_1_par1_text">
The lazy dog lay sleeping in the tall grass.
</text>
</par>
<par id="chapter_1_par2">
<sync:mediaref id="chapter_1_par2_audio" idref="audio" sync:selector="t=15, 23" />
<text id="chapter_1_par2_text">
Behind a thick tuft of straws the quick brown fox waited, its hindlegs tensed, ready to leap.
</text>
</par>
</seq>
</seq>
</body>
</smil>
<smil>
<head>
<sync:media type="application/lpf+zip" id="audiobook" src="https://audiobooks.com/titles/theexcitingbookofwords/theexcitingbookofwords.lpf" />
<sync:media type="application/zip+epub" id="epub" src="https://ebooks.com/titles/theexcitingbookofwords/theexcitingbookofwords.epub" />
</head>
<body>
<par id="cover" type="cover">
<image src="https://audiobooks.com/titles/theexcitingbookofwords/cover.png"
alt="Book title in large exciting font. The background is filled with a multi-colored flurry of words set in varied fonts." />
<mediaref idref="audiobook" sync:selector="t=0, 8" />
<mediaref idref="epub" sync:selector="epubcfi(/6/4![cover]/4[0:67])" />
</par>
<seq id="part_1" type="part">
<seq id="chapter_1" type="chapter">
<par>
<sync:mediaref idref="audiobook" sync:selector="t=10, 14" />
<sync:mediaref idref="epub" sync:selector="epubcfi(/6/6![chapter1]/4[0:49])" />
</par>
</seq>
</seq>
</body>
</smil>
This sample document can afford the reader similar features as described in the http://idpf.org/epub/renditions/region-nav document.
<smil>
<head>
<sync:media type="application/zip+epub" id="comic" src="https://mycomics.com/superduperhero/episode1.epub" />
<sync:media type="audio/mp4" id="comic-sfx" src="https://mycomics.com/superduperhero/episode1-sfx.mp4" sync:volume="0.7" />
<sync:media type="audio/mp4" id="comic-sfx-blam" sync:volume="0.7" clipBegin="233s" clipEnd="235s" src="https://mycomics.com/superduperhero/episode1-sfx.mp4" />
<sync:media type="audio/mp4" id="comic-sfx-wham" sync:volume="0.7" clipBegin="236s" clipEnd="238s" src="https://mycomics.com/superduperhero/episode1-sfx.mp4" />
<sync:media type="audio/mp4" id="comic-score" src="https://mycomics.com/superduperhero/episode1-score.mp4" sync:volume="0.9" />
<sync:media type="audio/mp4" id="comic-score-cover" idref="comic-score" clipBegin="1.5s" clipEnd="55s" repeatCount="indefinite" />
<sync:media type="audio/mp4" id="comic-narration" src="https://mycomics.com/superduperhero/episode1-narration.mp4" />
</head>
<body type="comicbook">
<par id="cover" type="cover">
<audio idref="comic-score-cover" sync:volume="0.7" />
<seq id="cover-main-panel" type="panel">
<par id="coverpanel-masthead">
<audio idref="comic-narration" clipBegin="10s" clipEnd="15s" />
<sync:mediaref idref="myComic" sync:selector="epubcfi(/6/4[cover]!/)" panZoom="150, 100, 400, 120" />
</par>
<par id="coverpanel-illustration">
<audio idref="comic-narration" clipBegin="16s" clipEnd="21s" />
<sync:mediaref idref="myComic"
sync:selector="epubcfi(/6/4[cover]!/4)"
sync:clipPath="M 49,58 H 146 C 154,58 160,64 160,72 V 114 C 160,121 154,128 146,128 H 49 C 41,128 3,121 35,114 V 72 C 35,64 41,58 49,58 Z"
panZoom="150, 100, 400, 120" />
</par>
</seq>
</par>
<par id="page1">
<sync:mediaref idref="soundtrack" fill="repeat" />
<seq>
<seq id="page1-panel1" type="panel">
<par id="page1-panel1-bubble1" type="bubble">
<audio idref="comic-narration" clipBegin="10s" clipEnd="15s" />
<sync:mediaref idref="myComic"
sync:selector="epubcfi(/6/4[page1]!/)"
sync:clipPath="M 43,65 H 189 V 157 H 43 Z"
panZoom="20, 20, 175, 120" />
</par>
<par id="page1-panel1-bubble1" type="bubble">
<audio idref="comic-narration" clipBegin="10s" clipEnd="15s" />
<sync:mediaref idref="myComic"
sync:selector="epubcfi(/6/4[page1]!/)"
sync:clipPath="M 43,65 H 189 V 157 H 43 Z"
panZoom="20, 20, 175, 120" />
</par>
</seq>
<seq id="page1-panel2" type="panel">
<par id="page1-panel2-bubble1" type="bubble">
<audio idref="comic-narration" clipBegin="10s" clipEnd="15s" />
<sync:mediaref idref="myComic"
sync:selector="epubcfi(/6/4[page1]!/)"
sync:clipPath="M 7,13 H 152 V 179 H 7 Z"
panZoom="20, 120, 80, 120" />
</par>
<par id="page1-panel2-bubble2" type="bubble">
<audio idref="comic-narration" clipBegin="10s" clipEnd="15s" />
<sync:mediaref idref="myComic"
sync:selector="epubcfi(/6/4[page1]!/)"
sync:clipPath="M 6,227 H 203 V 290 H 6 Z"
panZoom="20, 120, 175, 120" />
</par>
</seq>
</seq>
</par>
<body>
</smil>
Files are represented as cursive
- OCF
-
META-INF
- container.xml
- metadata.xml
- encryption.xml
- etc
-
my-sync-comic.smil
-
assets
- sounds
- text
-
<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
<rootfiles>
<rootfile full-path="my-sync-comic.smil"
media-type="application/smil+xml" />
</rootfiles>
</container>