Gemini Pro 1.5 Revolutionizes AI Video Content Analysis

Google introduced Gemini Pro 1.5, a huge upgrade to their Gemini series of AI models. You can test the model through Google AI Studio using text, video, code, audio, or images!

To showcase its power, Simon Willison provided it with a short video alongside a prompt “JSON array of books in this video.”

8 second video of bookshelf sent to Gemini.

From this 8 second video, Gemini told Simon Willison exactly what was on his bookshelf (a pretty good bookshelf by the way - Rework and Structure & Interpretation are favorites).

Gemini Pro 1.5 is a large leap in AI video content reading capabilities. Its new token context size allows it to analyze videos quickly and efficiently, extracting detailed information with minimal token expenditure. This is a big step forward, showing us what AI might do next in understanding and pulling out information from video content.

New Features & Demos

Meyda: Javascript Audio Feature Extraction

Meyda takes audio signals and analyzes them for various characteristics, outputting the results as numerical data. These outputs can be used to power a wide range of applications: from music identification and classification to audio-driven visualizations.

Feel free to play around with a live demo here. Have any ideas on how audio signal analysis can help your business? Let us know!

Off the top of our heads, language learning apps could integrate “accent coaching” for improved pronunciation tests.

Meyda live demo presents the extracted features of a sound signal
Meyda live demo presents the extracted features of a sound signal.

Thank You to All Our Applicants!

We recently opened a job post for a junior software engineer and received a great amount of candidates. We appreciate each and every one of you that took the time to apply. Everyone should hear back by the end of next week!

What Caught Our Eye

  • Sora - Creating Video from Text: OpenAI’s Sora hopes to revolutionize content creation by enabling the generation of videos directly from text inputs. Looking at their videos it’s hard to tell it is all AI generated!
  • Unexpected Bill from Netlify: An unexpected bill for over $100k from Netlify, a cloud service provider, serves as an important reminder to understand pricing models and set up usage alerts.
  • From Open Source to Revenue: Learn about EmailEngine successfully transitioning from an open-source project to a profitable business.
  • Finding AWS Account ID of Any S3 Bucket: This method, already reported to AWS, reveals the AWS account ID associated with any S3 bucket, emphasizing the significance for cloud security and data privacy.
  • Why We Keep Getting Phished: You might have received texts from FedEx about packages you are not aware of. This article covers this phishing incident, displaying some digital communication security issues.
  • GPT Trademark Application Failure: An attempt to trademark “GPT” by OpenAi was rejected.