Posts tagged google in Mar
Filters: Month: Mar × google × Sorted by date
Function calling with Gemma (via) Google's Gemma 3 model (the 27B variant is particularly capable, I've been trying it out via Ollama) supports function calling exclusively through prompt engineering. The official documentation describes two recommended prompts - both of them suggest that the tool definitions are passed in as JSON schema, but the way the model should request tool executions differs.
The first prompt uses Python-style function calling syntax:
You have access to functions. If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]
You SHOULD NOT include any other text in the response if you call a function
(Always love seeing CAPITALS for emphasis in prompts, makes me wonder if they proved to themselves that capitalization makes a difference in this case.)
The second variant uses JSON instead:
You have access to functions. If you decide to invoke any of the function(s), you MUST put it in the format of {"name": function name, "parameters": dictionary of argument name and its value}
You SHOULD NOT include any other text in the response if you call a function
This is a neat illustration of the fact that all of these fancy tool using LLMs are still using effectively the same pattern as was described in the ReAct paper back in November 2022. Here's my implementation of that pattern from March 2023.
Putting Gemini 2.5 Pro through its paces
There’s a new release from Google Gemini this morning: the first in the Gemini 2.5 series. Google call it “a thinking model, designed to tackle increasingly complex problems”. It’s already sat at the top of the LM Arena leaderboard, and from initial impressions looks like it may deserve that top spot.
[... 2,400 words]Notes on Google’s Gemma 3
Google’s Gemma team released an impressive new model today (under their not-open-source Gemma license). Gemma 3 comes in four sizes—1B, 4B, 12B, and 27B—and while 1B is text-only the larger three models are all multi-modal for vision:
[... 804 words]State-of-the-art text embedding via the Gemini API
(via)
Gemini just released their new text embedding model, with the snappy name gemini-embedding-exp-03-07. It supports 8,000 input tokens - up from 3,000 - and outputs vectors that are a lot larger than their previous text-embedding-004 model - that one output size 768 vectors, the new model outputs 3072.
Storing that many floating point numbers for each embedded record can use a lot of space. thankfully, the new model supports Matryoshka Representation Learning - this means you can simply truncate the vectors to trade accuracy for storage.
I added support for the new model in llm-gemini 0.14. LLM doesn't yet have direct support for Matryoshka truncation so I instead registered different truncated sizes of the model under different IDs: gemini-embedding-exp-03-07-2048, gemini-embedding-exp-03-07-1024, gemini-embedding-exp-03-07-512, gemini-embedding-exp-03-07-256, gemini-embedding-exp-03-07-128.
The model is currently free while it is in preview, but comes with a strict rate limit - 5 requests per minute and just 100 requests a day. I quickly tripped those limits while testing out the new model - I hope they can bump those up soon.
Career Update: Google DeepMind -> Anthropic. Nicholas Carlini (previously) on joining Anthropic, driven partly by his frustration at friction he encountered publishing his research at Google DeepMind after their merge with Google Brain. His area of expertise is adversarial machine learning.
The recent advances in machine learning and language modeling are going to be transformative [d] But in order to realize this potential future in a way that doesn't put everyone's safety and security at risk, we're going to need to make a lot of progress---and soon. We need to make so much progress that no one organization will be able to figure everything out by themselves; we need to work together, we need to talk about what we're doing, and we need to start doing this now.
llm-gemini 0.1a1. I upgraded my llm-gemini plugin to add support for the new Google Gemini Pro 1.5 model, which is beginning to roll out in early access.
The 1.5 model supports 1,048,576 input tokens and generates up to 8,192 output tokens—a big step up from Gemini 1.0 Pro which handled 30,720 and 2,048 respectively.
The big missing feature from my LLM tool at the moment is image input—a fantastic way to take advantage of that huge context window. I have a branch for this which I really need to get into a useful state.
900 Sites, 125 million accounts, 1 vulnerability (via) Google’s Firebase development platform encourages building applications (mobile an web) which talk directly to the underlying data store, reading and writing from “collections” with access protected by Firebase Security Rules.
Unsurprisingly, a lot of development teams make mistakes with these.
This post describes how a security research team built a scanner that found over 124 million unprotected records across 900 different applications, including huge amounts of PII: 106 million email addresses, 20 million passwords (many in plaintext) and 27 million instances of “Bank details, invoices, etc”.
Most worrying of all, only 24% of the site owners they contacted shipped a fix for the misconfiguration.
Google Scholar search: “certainly, here is” -chatgpt -llm (via) Searching Google Scholar for “certainly, here is” turns up a huge number of academic papers that include parts that were evidently written by ChatGPT—sections that start with “Certainly, here is a concise summary of the provided sections:” are a dead giveaway.
If you ask Microsoft’s Bing chatbot if Google’s Bard chatbot has been shut down, it says yes, citing as evidence a news article that discusses a tweet in which a user asked Bard when it would be shut down and Bard said it already had, itself citing a comment from Hacker News in which someone joked about this happening, and someone else used ChatGPT to write fake news coverage about the event.
Don’t trust AI to talk accurately about itself: Bard wasn’t trained on Gmail
Earlier this month I wrote about how ChatGPT can’t access the internet, even though it really looks like it can. Consider this part two in the series. Here’s another common and non-intuitive mistake people make when interacting with large language model AI systems: asking them questions about themselves.
[... 1,950 words]Here are some absurdly expensive things you can do on a trip to Tokyo: Buy a golden toilet. There is a toilet in Tokyo that is made of gold and costs around 10 million yen. If you are looking for a truly absurd experience, you can buy this toilet and use it for your next bowel movement. [...]
Google Bard is now live. Google Bard launched today. There’s a waiting list, but I made it through within a few hours of signing up, as did other people I’ve talked to. It’s similar to ChatGPT and Bing—it’s the same chat interface, and it can clearly run searches under the hood (though unlike Bing it doesn’t tell you what it’s looking for).
google-cloud-4-words. This is really useful: every Google Cloud service (all 250 of them) with a four word description explaining what it does. I’d love to see the same thing for AWS. UPDATE: Turns out I had—I can’t link to other posts from blogmark descriptions yet, so search “aws explained” on this site to find it.
Why Google MapMaker is not Open. Non-commercial use only, strict attribution requirements and you aren’t allowed to use the data for services that might compete with Google. This is why I’m disappointed every time I see Google encouraging people to contribute to Map Make, especially in the developing world—if those people contributed to OpenStreetMap instead they would be building something far more valuable for their community.
RE2: a principled approach to regular expression matching. Google have open sourced RE2, the C++ regular expression library they developed for Google Code Search, Sawzall, Bigtable and other internal projects. Unlike PCRE it avoids the potential for exponential run time and unbounded stack usage and guarantees that searches complete in linear time, mainly by dropping support for back references.
Apparently [unladen-swallow] is already 30% faster than CPython, and this version is being used to run some of the Python code on YouTube.
ProjectPlan—unladen-swallow. A branch of Python 2.6 aiming to radically improve performance (the target is a 5x improvement), by compiling Python to machine code using LLVM’s JIT engine. I think this is a Google 20% time project (or maybe not, see the comments). An early version without LLVM is already available for download.
django-gae2django. An implementation of the Google App Engine API (datastore, memcache, urlfetch, users and mail) that runs on Django, allowing you to take an existing application written for App Engine and deploy it on your own server on top of Django.
An OpenSocial Foundation. “Today we are pleased to announce that Google is joining together with Yahoo! and MySpace in the creation of a non-profit foundation for the open and transparent governance of the OpenSocial specifications and intellectual property.” Good move; I’d personally love to see this happen with Google Gears.
Introducing the Google Contacts Data API. Brilliant! (and about time)—now there’s no excuse for asking your users for their Gmail username and password so you can import contacts from their address book. Yahoo! and Microsoft need to catch up on this one fast.
The real reason Google’s clicks are flat. Rich Skrenta explains that Google’s recent reduction of the clicable area in Adsense ads, while reducing click-throughs by 60%, will eventually balance out due to non-accidental click-throughs being worth more to advertisers.
How to beat Google, part 1. Rich Skrenta with 12 steps to taking on Google in the search engine space, including some great insights in to smart UI design.
New Open Source Utility Library for the Google Maps API (via) Google are taking a hybrid approach to development on their Maps API—an open source utility library layered on top of their closed source, obfuscated core code.
My photos tagged “cheese” on a Google Map. You can paste a Flickr GeoRSS feed directly in to the Google Maps query box.
KML and GeoRSS support added to the Google Maps API. Since Flickr can output GeoRSS, this means you can now plot your Flickr photos on a Google Map (if you’re so inclined).
Two visions. It looks like Mark Pilgrim is going to be joining Hixie at Google.
Google Video: How do I enter transcripts? Neat feature of Google Video I hadn’t seen before: you can upload timestamped transcripts of your videos. Anyone seen a video that uses these?
Google Seattle conference on scalability. Google are hosting a conference on scalability in Seattle on June 23rd. They’ve just put out the CfP.
Thirty five year old cookies
I’m finding myself slightly confused about the Google backlash washing around the blogosphere, which is summarised quite well by Gavin Sheridan. Most of the arguments against using Google unsurprisingly centre around privacy issues, in particular the “35 year cookie”. I was under the impression that cookies could only be set for a maximum of a year, but having checked Netscape’s Cookie Specification and RFC 2965 it appears I was mistaken.
[... 566 words]