Video encoding/decoding in Java

This is a new forum that can be used to discuss anything that's not directly related to JPatch.

Video encoding/decoding in Java

Postby sascha » Wed Mar 25, 2009 3:56 pm

Following our discussion in this thread...

I've started to write a small Java wrapper that executes mencoder from withing a Java application and uses the process' stdin/stdout to communicate with it. It works both ways, for decoding and playback (by using mencoder to read any format, transcode it to "raw" and pass the raw frames via stdin to the java application) and encoding (by passing raw frames from java to mencoder's stdin and encoding them to any format).

While this approach works, I ran across something much more interesting while reading the Wikipedia article about JMF: Xuggle. It's an (object oriented) Java framework that uses native ffmpeg code under the hood to decode and encode just about any media format. I haven't tested it yet, but it seems to be under active development (latest stable release is from 17-Mar-2009) and the descriptions on their web-page sound very promising. And it's GPL'd too...
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Wed Mar 25, 2009 5:26 pm

sascha wrote:While this approach works, I ran across something much more interesting while reading the Wikipedia article about JMF: Xuggle.

Cool! I'll have to look at it. One of the users is complaining that GlueFace is displaying the frames negative with some odd clipping. It would be nice to ditch my buggy code for something more robust. :)
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby dcuny » Wed Mar 25, 2009 6:51 pm

Xuggler looks cool, but it doesn't look like it's usable by anyone who's not a developer. I'm assuming that the end users for GlueFace are technologically illiterate - if there's not a "one button to install" option available on all platforms, it's not likely they will be able to use it. :(
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Thu Mar 26, 2009 11:40 am

Yes, you have to install the native libraries in some place the JVM can find them (i.e. somewhere inside java.library.path).

JPatch has a temporary solution for that (the native JOGL code) - it checks if the libraries can be found and if not will copy them to a location the user selects (which could mean that the user has to run JPatch as root or administrator once to do the necessary installation).

The most user friendly way of course would be a native installer that installs the application, possibly a JVM and all the required native libraries. Such solutions exist (even as free software), at least for Windows. I have to look into this and test the various solutions to find the one that works best for JPatch (and possibly for OSX as well) - if you too need such an installer and have some time to look into this it would be great of course :wink:

For Linux I think the best way to make installation easy are .deb and .rpm packages (with dependencies set on a JRE installation and the required native libraries)
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Thu Mar 26, 2009 6:09 pm

sascha wrote:I have to look into this and test the various solutions to find the one that works best for JPatch (and possibly for OSX as well) - if you too need such an installer and have some time to look into this it would be great of course.

Yes, it would! :) Installation of JPatch is pretty painless, and having something like that for Xuggler would be great.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Thu Mar 26, 2009 7:22 pm

If you'd like to incorporate my solution, here's the relevant class (NativeLibraryHelper):
Code: Select all
package jpatch.auxilary;

import java.awt.Frame;
import java.io.*;
import java.util.*;
import java.security.*;

import javax.swing.JComboBox;
import javax.swing.JFrame;

import jpatch.boundary.ui.JPatchDialog;

public class NativeLibraryHelper {
   public static String NATIVE_LIBS_DIR = "nativelibs/";
   
   public static enum Os {
      UNKNOWN_OTHER,
      WINDOWS,
      LINUX,
      MAC_OS_X,
      SOLARIS
   }
   
   public static enum Arch {
      UNKNOWN_OTHER,
      X86,
      AMD64,
      PPC,
      SPARC
   }
   
   private static String getNativeLibsDir(Os os, Arch arch) {
      String osDir = null, archDir = null;
      switch(os) {
      case WINDOWS:
         osDir = "windows/";
         break;
      case LINUX:
         osDir = "linux/";
         break;
      case MAC_OS_X:
         osDir = "osx/";
         break;
      }
      switch (arch) {
      case X86:
         archDir = "x86/";
         break;
      case AMD64:
         archDir = "amd64/";
         break;
      case PPC:
         archDir = "ppc/";
         break;
      }
      
      /* windows hack */
      if (os == Os.WINDOWS) {
         archDir = "x86/";
      }
      
      if (osDir != null && archDir != null) {
         return NATIVE_LIBS_DIR + osDir + archDir;
      }
      
      return null;
   }
   
   private static String[] getLibraryNames(Os os) {
      List<String> libList = new ArrayList<String>();
      libList.add(System.mapLibraryName("jogl"));
      libList.add(System.mapLibraryName("jogl_cg"));
      libList.add(System.mapLibraryName("jogl_awt"));
      if (os == Os.LINUX) {
         libList.add(System.mapLibraryName("jogl_drihack"));
      }
      return libList.toArray(new String[libList.size()]);
   }
   
   private static byte[] digest(InputStream in, String algorithm) throws NoSuchAlgorithmException, IOException {
      byte[] buffer = new byte[4096];
      MessageDigest digest = MessageDigest.getInstance(algorithm);
      int bytes = 0;
      while ((bytes = in.read(buffer)) > -1) {
         digest.update(buffer, 0, bytes);
      }
      in.close();
      return digest.digest();
   }
   
   private static Os detectOs() {
      String osName = System.getProperties().getProperty("os.name");
      if (osName.startsWith("Windows")) {
         return Os.WINDOWS;
      } else if (osName.equals("Linux")) {
         return Os.LINUX;
      } else if (osName.equals("Mac OS X")) {
         return Os.MAC_OS_X;
      } else if (osName.equals("Solaris")) {
         return Os.SOLARIS;
      } else {
         return Os.UNKNOWN_OTHER;
      }
   }
   
   private static Arch detectArch() {
      String osArch = System.getProperties().getProperty("os.arch");
      if (osArch.startsWith("x86") || osArch.equals("i386") || osArch.equals("i586") || osArch.equals("i686")) {
         return Arch.X86;
      } else if (osArch.equals("ppc") || osArch.equals("PowerPC")) {
         return Arch.PPC;
      } else if (osArch.equals("sparc")) {
         return Arch.SPARC;
      } else if (osArch.equals("amd64")) {
         return Arch.AMD64;
      } else {
         return Arch.UNKNOWN_OTHER;
      }
   }
   
   public boolean checkLibraries(Frame owner) throws NoSuchAlgorithmException, IOException {
      String algorithm = "SHA-1";
      Os os = detectOs();
      Arch arch = detectArch();
      System.out.println("Platform identified as " + os + " " + arch);
      
      String dir = getNativeLibsDir(os, arch);
      String[] libs = getLibraryNames(os);
      String javaLibraryPath = System.getProperty("java.library.path");
      String[] folders = javaLibraryPath.split(System.getProperty("path.separator"));

      /* create digests for bundled libraries */
      byte[][] digests = new byte[libs.length][];
      for (int i = 0; i < libs.length; i++) {
         System.out.println(dir + ":" + libs[i]);
         digests[i] = digest(ClassLoader.getSystemResourceAsStream(dir + libs[i]), algorithm);
         System.out.println(libs[i] + " " + digestToString(digests[i]));
      }
      
      int libraryFolder = -1;
      /* scan folders in library path */
      scanFolders:
      for (int i = 0; i < folders.length; i++) {
         String folder = folders[i];
         for (int j = 0; j < libs.length; j++) {
            String lib = libs[j];
            File libFile = new File(folder, lib);
            if (libFile.exists()) {
               /* set the libraryFolder to the path index where the first library was found */
               System.out.println(lib + " exists in " + folder);
               libraryFolder = i;
               break scanFolders;
            }
         }
      }
      
      if (libraryFolder == -1) {
         // no libraries were found, prompt user to install in any library path folder
         String message = "<b>The required JOGL native libraries have not been found. Please specify the folder " +
               "of the library path where JPatch should try to install the required libraries.</b>" +
               "<p>If the libraies have already been installed or you wish to install them to a different " +
               "folder, please add this folder to the library path and restart JPatch with the " +
               "<code>-Djava.library.path=<i>&lt;path&gt;</i></code> commandline switch.";
         JComboBox folderCombo = new JComboBox(folders);
         int selection = JPatchDialog.showDialog(owner, "JOGL native libraries installation", JPatchDialog.WARNING, message, folderCombo, new String[] { "Install", null, "Quit" }, 1, "400");
         if (selection != 0) {
            /* Quit */
            System.exit(0);
         }
         /* install to specified folder */
         try {
            for (int i = 0; i < libs.length; i++) {
               String lib = libs[i];
               InputStream source = ClassLoader.getSystemResourceAsStream(dir + lib);
               File destination = new File(folders[folderCombo.getSelectedIndex()], lib);
               install(source, destination);
            }
         } catch (IOException e) {
            message = "<b>An error occured during the installation of the JOGL native libraries:</b><p>" +
                  "<font color='red'>" + e.getMessage() + "</font>" +
                  "<p>You possibly need administrator (root) privileges to install files in the specified folder.";
            JPatchDialog.showDialog(owner, "JPatch error", JPatchDialog.ERROR, message, null, new String[] { "OK" }, 0, "300");
         }
         return false;
      } else {
         List<Integer> reinstall = new ArrayList<Integer>();
         List<Integer> install = new ArrayList<Integer>();
         String folder = folders[libraryFolder];
         /* scan library files in library folder */
         for (int i = 0; i < libs.length; i++) {
            String lib = libs[i];
            File libFile = new File(folder, lib);
            if (libFile.exists()) {
               if (!Arrays.equals(digests[i], digest(new FileInputStream(libFile), algorithm))) {
                  reinstall.add(i);
               }
            } else {
               install.add(i);
            }
         }
         if (reinstall.size() == 0 && install.size() == 0) {
            // all files found, all hashes ok, we can continue...
            return true;
         } else {
            // some files need to be re-installed (and maybe some need to be installed)...
            String message, buttonText;
            if (install.size() == 0) {
               // all files need to be re-installed
               message = "<b>The required JOGL native libraries have been found in " +
                     folder + ", but they appear to be corrupt or of an unsupported version " +
                     "and need to be re-installed.</b>" +
                     "<p>Overwriting these files with the libraries from JOGL JSR-231 version 1.0.0 " +
                     "may break other applications that depend on the installed version. If you do not " +
                     "want to overwrite these files, select QUIT and restart JPatch with another " +
                     "library-path by setting the " +
                     "<code>-Djava.library.path=<i>&lt;path&gt;</i></code> commandline switch." +
                     "<p>Choosing RE-INSTALL will overwrite the following files in " + folder + ":<ul>";
               for (int i = 0; i < libs.length; i++) {
                  if (reinstall.contains(i)) {
                     message += "<li>" + libs[i] + "</li>";
                  }
               }
               message += "</ul>";
               buttonText = "Re-Install";
            } else if (reinstall.size() == 0) {
               message = "<b>The required JOGL native libraries have been found in " +
                     folder + ", but some files appear to be missing and need to be installed.</b>" +
                     "<p>If you do not " +
                     "want to install the missing files in this folder, select QUIT and restart JPatch with another " +
                     "library-path by setting the " +
                     "<code>-Djava.library.path=<i>&lt;path&gt;</i></code> commandline switch.";
               buttonText = "Install";
            } else {
               message = "<b>Some of the required JOGL native libraries have been found in " +
                     folder + ", but they appear to be corrupt or of an unsupported version " +
                     "and need to be re-installed. Some files are missing and need to be installed.</b>" +
                     "<p>Overwriting the already installed files with the libraries from JOGL JSR-231 version 1.0.0 " +
                     "may break other applications that depend on the installed files. If you do not " +
                     "want to overwrite these files, select QUIT and restart JPatch with another " +
                     "library-path by setting the " +
                     "<code>-Djava.library.path=<i>&lt;path&gt;</i></code> commandline switch." +
                     "<p>Choosing RE-INSTALL will overwrite the following files in " + folder + ":";
               for (int i = 0; i < libs.length; i++) {
                  if (reinstall.contains(i)) {
                     message += "<li>" + libs[i] + "</li>";
                  }
               }
               message += "</ul>";
               buttonText = "Re-Install";
            }
            int selection = JPatchDialog.showDialog(owner, "JOGL native libraries installation", JPatchDialog.WARNING, message, null, new String[] { buttonText, null, "Quit" }, 1, "400");
            if (selection != 0) {
               /* Quit */
               System.exit(0);
            }
            /* re-install */
            try {
               for (int i = 0; i < libs.length; i++) {
                  String lib = libs[i];
                  InputStream source = ClassLoader.getSystemResourceAsStream(dir + lib);
                  File destination = new File(folder, lib);
                  install(source, destination);
               }
            } catch (IOException e) {
               message = "<b>An error occured during the installation of the JOGL native libraries:</b><p>" +
                     "<font color='red'>" + e.getMessage() + "</font>" +
                     "<p>You possibly need administrator (root) privileges to install files in the specified folder.";
               JPatchDialog.showDialog(owner, "JPatch error", JPatchDialog.ERROR, message, null, new String[] { "OK" }, 0, "300");
            }
            return false;
         }
      }
   }
   
   private void install(InputStream source, File destination) throws IOException {
      OutputStream out = new FileOutputStream(destination);
      byte[] buffer = new byte[4096];
      int bytes = 0;
      while ((bytes = source.read(buffer)) > -1) {
         out.write(buffer, 0, bytes);
      }
      source.close();
      out.close();
   }
   
   private static String digestToString(byte[] digest) {
      StringBuilder sb = new StringBuilder();
      for (int i = 0; i < digest.length - 1; i++) {
         sb.append(Integer.toHexString(digest[i] & 0xff));
      }
      return sb.toString();
   }
   
   /** for testing */
   public static void main(String[] args) throws Exception {
      JFrame frame = new JFrame();
      frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      frame.setSize(800, 600);
      frame.setVisible(true);
      
      while (!new NativeLibraryHelper().checkLibraries(frame));
      System.out.println("OK");
      System.exit(0);
   }
   
}

the only dependency is my improved dialog, JPatchDialog:
Code: Select all
/*
* $Id:$
*
* Copyright (c) 2005 Sascha Ledinsky
*
* This file is part of JPatch.
*
* JPatch is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* JPatch is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with JPatch; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
*/
package jpatch.boundary.ui;

import java.awt.*;
import java.awt.event.*;

import javax.swing.*;
import javax.swing.text.*;


/**
* @author sascha
*
*/
@SuppressWarnings("serial")
public class JPatchDialog extends JDialog {
   public final static Icon ERROR = new ImageIcon(ClassLoader.getSystemResource("jpatch/images/icons_64x64/dialog-error.png"));
   public final static Icon INFORMATION = new ImageIcon(ClassLoader.getSystemResource("jpatch/images/icons_64x64/dialog-information.png"));
   public final static Icon WARNING = new ImageIcon(ClassLoader.getSystemResource("jpatch/images/icons_64x64/dialog-warning.png"));
   public final static Icon QUESTION = new ImageIcon(ClassLoader.getSystemResource("jpatch/images/icons_64x64/dialog-question.png"));
   
   private int selectedOption = -1;
   private Font font = new Font("SansSerif", Font.PLAIN, 12);
   
   private JPatchDialog(Frame owner, String title, boolean modal, Icon icon, String message, Component component, String[] options, int focus, String width) {
      super(owner, title, modal);
      setLayout(new BorderLayout());
      Box buttonBox = Box.createHorizontalBox();
      if (icon != null) {
         JLabel iconComponent = new JLabel(icon);
         iconComponent.setBorder(BorderFactory.createEmptyBorder(16, 16, 16, 0));
         add(iconComponent, BorderLayout.WEST);
      }
      Box textBox = Box.createVerticalBox();
      
      final JEditorPane text = new JEditorPane("text/html", "<div width='" + width + "'>" + message + "</div>");
      text.putClientProperty(JEditorPane.HONOR_DISPLAY_PROPERTIES, true);
      text.getDocument().putProperty(PlainDocument.lineLimitAttribute, 20);
      text.setFont(font);
      text.setEditable(false);
      text.setBorder(BorderFactory.createEmptyBorder(16, 16, 16, 16));
      text.setOpaque(false);
      textBox.add(text);
      if (component != null) {
         JPanel panel = new JPanel();
         panel.setLayout(new BorderLayout());
         panel.add(component, BorderLayout.CENTER);
         panel.setBorder(BorderFactory.createEmptyBorder(0, 16, 16, 16));
         textBox.add(panel);
      }
      textBox.add(buttonBox);
      
      add(textBox, BorderLayout.CENTER);
      
      Component focusComponent = null;
      int o = 0;
      for (String option : options) {
         if (option != null) {
            JButton button = new JButton(option);
            buttonBox.add(Box.createHorizontalStrut(8));
            buttonBox.add(button);
            buttonBox.add(Box.createHorizontalStrut(8));
            if (o == focus)
               focusComponent = button;
            final int opt = o;
            button.addActionListener(new ActionListener() {
               public void actionPerformed(ActionEvent e) {
                  selectedOption = opt;
                  dispose();
               }
            });
            o++;
         } else {
            buttonBox.add(Box.createHorizontalGlue());
         }
      }
      buttonBox.setBorder(BorderFactory.createEmptyBorder(0, 8, 16, 8));
      pack();
      setResizable(false);
      if (focusComponent != null) {
         focusComponent.requestFocus();
      }
      setLocationRelativeTo(owner);
      setVisible(true);
   }

   /**
    * Displays a dialog window
    * @param owner   the frame owning this dialog
    * @param title   a title string
    * @param icon the icon to display
    * @param message a message to display
    * @param component a component to display
    * @param options an array of (string) options to show - may contain nulls, which will put separators between the options
    * @param focus which option should have the focus (nulls are not counted)
    * @param width the width of the message-box (in html syntax, e.g. 100px)
    * @return the index of the selected option (nulls are not counted), or -1 if the window was closed
    */
   public static int showDialog(Frame owner, String title, Icon icon, String message, Component component, String[] options, int focus, String width) {
      JPatchDialog dialog = new JPatchDialog(owner, title, true, icon, message, component, options, focus, width);
      return dialog.selectedOption;
   }
}


You also need the icons JPatchDialog refers to - I've used the Tango icons, grab them from tango or from the JPatch svn.
There's only rudimentary documentation, but it should be pretty obvious what's going on. It searches for native libraries in the java.library.path, if found compares their hashes with those stored in the jar file, and then either returns or prompts the user what to do.

The native libraries must be part of the jar file (or the eclipse project), in Eclipse under
src/nativelibs/OS/ARCH where OS can be linux, osx or windows and ARCH can be x86, amd64 or ppc

Hope this is useful.

I'd still rather see a full "Installer", at least for Windows, for the final version of JPatch - there are a few such installer-creators available, but I haven't found the time to truly test them all yet.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby sascha » Sat Mar 28, 2009 10:29 am

I gave xuggler a try, and you're right, the bundles that can be downloaded are not for end-users. I've been able to compile it on Linux, but it creates a lot of libraries and tries to install them to some system folders (like /usr/local/lib, etc.) - while I'm fine with that, it's actually not usable for bundling xuggler with an application. Also, downloading over 30MB just for the xuggler part seems a bit much. I'll invest some more time into xuggler, but, in parallel, also follow the mencoder approach - this too requires the end-user to install mplayer/mencoder first, but that's rather straight-forward and these applications can be used for other things too, which perhaps justifies the large download for some.

I've tested "reading" from mencoer (i.e. playback), and it works just fine. I've tested playback with a few 1280x720 HD videos, which - depenging on the codec - mplayer can playback with about 40 to 60 fps, and with my "bridge to java" inbetween it still plays with 30 to 50 fps.
It uses mencoder to read the movie file and output the raw images via stdout. Java then reads the raw images from stdin into the byte-array of an awt BufferedImage, which is displayed in a JFrame. I'll put the source online one it's a little more useable.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Sun Mar 29, 2009 12:27 am

sascha wrote:I've tested "reading" from mencoer (i.e. playback), and it works just fine. I've tested playback with a few 1280x720 HD videos, which - depending on the codec - mplayer can playback with about 40 to 60 fps, and with my "bridge to java" inbetween it still plays with 30 to 50 fps.

That sounds great! I'll try adding it to GlueFace when it's a ready for prime time. I might even consider dusting off my crappy little Java video editor... :)
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Sun Mar 29, 2009 9:05 am

I've now made it multi-threaded and it's even faster. It cat read frames at more than 60fps of a 1280x720 h264 encoded video, it's nearly as fast as playback with mplayer alone. It's difficult to lock it to a fixed framerate (e.g. 24p) though, it works and looks OK, but occasionally there's some stuttering, at least when using Java2D for drawing the frames (perhaps it gets better when using GL for drawing). The way it's implemented now I also can't navigate through the video (I could pause, but not seek forwards or backwards) - but that's not the main application anyway. I thought it as a way to read video files, frame by frame, then do something with that frames, and write them back to a different video file - all the encoding and decoding is done by mplayer/mencoder behind the scenes, and it works in realtime or faster (OK, for Full-HD you'd need a pretty fast machine).
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Mon Mar 30, 2009 9:42 am

GlueFace doesn't allow playback, per se. You can scrub through a timeline and playback frames, but there's no "Play" button.

I've got my compositing program lying around. It's got a number of classes for playing around with various color spaces, substituting colors, applying focal blur, adding subtitles (using your code), "melting" frames, iris zooms, film scratches, and stuff like that. It might be nice to package that up into something usable.

I've also got my Flipbook project, which was designed to be a small video editor. I'm not sure there's really a need for such a thing, though. There are a number of free programs in Linux and Windows that already do that. The idea was that you could add the transitions (and subtitles) from the compositing program, which isn't anything close to real time.

How does it handle sound?
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Mon Mar 30, 2009 1:52 pm

At the moment it doesn't support sound at all, but I have a few ideas.

Right now I'm just playing with the code, I'm not sure how useful it will be.
My goal is to have something like imp-edit but with a GUI. And instead of reading and writing just single image files, it should be able to read from any source, and directly encode into any video format.

The idea is pretty simple: There are video sources (that provide a sequence of BufferedImages) and video destinations (where you can write BufferedImages into). You can also have something that does both in between to add effects.

One of the main reasons is that I don't want to output single frames into some temporary directory, I want to encode directly after editing. Then I can't memorize all those mencoder commandline options, so some GUI that allows to choose at least the most common codecs and settings and stores them somewhere would be nice as well.

I've written some image-processing tools just for fun too, and for small resolutions (640x480 or less) most of them even work in realtime (blur, LCD or TV-scanline effects, etc.)
One of the effects looks pretty cool, and I'm using it for a "special edition" of The Impostor which I plan to release this Wednesday, so stay tuned and check the front page.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Mon Mar 30, 2009 6:44 pm

sascha wrote:One of the effects looks pretty cool, and I'm using it for a "special edition" of The Impostor which I plan to release this Wednesday, so stay tuned and check the front page.

Cool! I'm looking forward to seeing it. :)
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Wed Apr 01, 2009 1:56 am

Check out the front page. A non youtubed version can be found here.
Enjoy 8)
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Video encoding/decoding in Java

Postby dcuny » Wed Apr 01, 2009 8:54 pm

That cracked me up! Especially [i]"The fake Rusty's performance does not convince the clerk." :mrgreen:

Of course, don't let John see it, or he'll argue that it's proof IRTC videos are better without sound. :wink:

You did all those effects yourself? There's a lot of cool stuff going on with the "aged film" look:

  • The borders are dirty, as if there were dust trapped around the lens;
  • There's a falloff to black as the image approaches the edges of the frame;
  • The frame drifts around;
  • There's a "bloom" effect;
  • The light level changes from frame to frame;
  • The image cycles in and out of focus slightly;
  • There are random "dust" specks;
  • There are long scratches across the frames.
Did I miss anything?
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Video encoding/decoding in Java

Postby sascha » Thu Apr 02, 2009 10:53 am

That cracked me up! Especially "The fake Rusty's performance does not convince the clerk."

I'm glad you like it. :-)
In my recollection silent movie intertitles don't use so much dialog, but rather explain what happens or is being said - so I tried to eliminate parts of the dialog.
Of course, don't let John see it, or he'll argue that it's proof IRTC videos are better without sound.

I thought that it's a pity that I hadn't had the idea earlier - it would have been a cool way to interpret the "no sound" rule.
You did all those effects yourself?

Yep. It all started with a quite innocent to-grayscale filter. I added a bit of noise and it suddenly started to look like some very old film - so I quickly hacked in some more effects until it looked very convincing. Here's a list of all the effects:
  • Color to grayscale conversion
  • boost contrast and brightness
  • add random noise
  • random overall brighness adjustments
  • vignette effect - darken the image at the corners, simulates a cheap (or for that matter vintage) lens.
  • randomly move image to simulate frame drift
  • add a dusty frame border
  • add random specks
  • add random vertical scratches
  • blur image (actually it's just a very simply 2x2 blur filter)
  • last not least: it's played back at 30fps instead of the original 24fps - this simulates the effect of playing old movies typically recorded at 18fps with TV framerates (23.97 or 25 fps)
I think that's all. The random factors for brightness changes and frame drift are constrained, so that the delta to the previous frame doesn't get too large. The vertical scratch positions actually don't use random but a sine-wave plus some harmonics.

There's no change in the overall blurring - I thought about blurring more in the corners or change the blur-filter size over time, but it turned out that it wasn't necessary (and it would make the filter more complex and slower) - the random change in overall brightness seems to also affect out perception of blurring.

The nice thing about the filter is that it runs in realtime (at least for low resolution material) - I'll put it online soon.

What completely baffled me was how well the piano piece fit to the images. I just did a Google search for royalty free silent-movie scores, and this was the second piece on the first link I checked. All I had to do was to add 0.5 seconds to one title-card to sync the clerks "oh" face (when he sees the Terminator) to the music, the rest was just coincidence (and not too little, just think about it: The music perfectly matches the "until..." intertitle, the argument, the "I'll be back" intertitle, the clerks reaction and the credits).

I've also posted a "confession" to the IRTC newsgroups here

PS: My first idea for an April Fool's joke was to announce a new version of JPatch and then link to Never Gonna Give You Up, but that's sort of dated (and I prefer rick-rolling people I don't like ;-) ). And of course I enjoyed doing something more creative again - it's definitely time to continue JPatch development and make some new animations with it.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Next

Return to Off topic

Who is online

Users browsing this forum: No registered users and 1 guest

cron