* A simple, flexible SQL client
* A notebook interface that lets you drag stuff around and view things side-by-side
* Query CSVs, JSON, and XLSX documents fully in-browser with Emscripten + SQLite
* Postgres, MySQL and BigQuery through a local connection bridge. Your data never touches a third party server.
I’ll be here with @antimatter15 for the rest of the morning answering questions!
This is really cool :) I nearly worked on such a project myself, out of frustration for simple solutions. I've seen similar things recently with SQL Notebook, Zeppelin, Jupyter, Flyspeed SQL query.. and more complex desktop apps like Tableau, QlikView, etc. I kept thinking to myself... why isn't there just a lighter weight query only tool. It helps people to learn, it's a handy utility, it helps you just use SQL as a utility for quick tasks using a UI - very nice.
A few questions/comments:
- Are you planning to put this in an Electron app? I think the more you can make it feel like an actual desktop app, the more likely you are to get more "regular" users. Just my opinion.
- While I love the simplicity of making this sound like a cloud service, I think that the nature of it being provided through the user's own infrastructure should be emphasized even more. As far as I can tell (and I haven't looked extensively), there is nothing that is actually "cloud oriented" about your service. Is it because people make there own "cloud" out of your bridge adapter? For people who want these tools on-premise and under their own control, you have a solution.
I fell in love w/ a fantastic little CLI util called 'lnav' which has an embedded sqlite db and lets you do all kinds of querying (eg write SQL queries against a few million rows of server logs in a custom log format, or use regex to transform files to/from arbitrary formats...). http://lnav.org
Hello! This is a great product, I gave it a spin in a local setup with some actual production data. I'll compare it to Metabase (even though you aren't trying to do the same thing).
* I really like the editor. It beats metabases ACE editor anyday.
* Here's the line graph comparison between the two: https://imgur.com/a/gd5P8 I think that metabase wins here because I can get more information off that graph rather than an area under the graph kinda thing that Franchise is trying to do. Is it configurable somewhere?
If you press the "Download" button, you can download the document as an HTML file. Share it through email or any other means, and it'll open in the Franchise web app— with all the queries, the results, and visualizations.
This looks great! Is there any chance that it will work with MS SQL Server? Sadly (i.e. over my recommendations) my company uses it rather than PostgreSQL.
same here - actually, today we use zeppelin, which is kind of a pain to setup on a cluster.
I see that you are using Reactjs. If you can build something that can connect to spark and python, I will pay for this and so will lots of people.
One of the big unsolved issues where we want to really throw money is have a notebook that my analysts can work on and then I can take straight to production and have it run multi-user.
There are lots of startups attempting to do this on top of Jupyter, but I believe building this from scratch is the right way.
I'm literally crying right now for a reactjs + d3js dashboard running on pyspark that multiple people can use simultaneously.
Tried running it locally and it's one of the most pleasant experiences I've had in recent memory. Extremely short number of steps and no hiccups. I already had something running on port 3000 and it even had a Y/N flag asking me if I wanted to run it on a different port ... I was like wow!
Great job OP and I'll be sure to try this out as we are having lots of difficulties managing data at my workplace.
I think this is a really clean product but I have a couple of questions.
1. I'm dealing with a dataset with >10M rows. Is there any way to kill a query that is currently running? I noticed at least with postgres that a long running query would stop me from performing any other queries.
2. The hashtag notation is very nice but would it be possible to cache the results of the hashtag without having to run the same query over again. This would more closely resemble the behavior of jupyter notebook, where results in previous cells can be recalculated by running those cells, but can be quickly accessed without recalculation.
For existing SQL tools there's currently basically two approaches: where either the full application is hosted, or where the full application is local.
Jupyter and Zeppelin generally fall into the second category— you have to set up Docker (or the myriad of dependencies), and edit a bunch of configuration files to launch a local server before getting started. If someone sends you a notebook, and you just want to read its contents and see the interactive charts, you still have to go through this ordeal.
On the other hand, there's Mode, Redash, and PopSQL, which are hosted solutions. Editing configuration files is replaced with a one-time setup and registration, but you have to entrust these startups with access to all your data.
Franchise takes a hybrid approach— it's a hosted static web page that contains all of the display logic, so if you save a notebook and share it with a friend, you can open it and it just works. To connect to a database, you start a local bridge which tunnels data from Franchise directly to your database through your computer.
We spent a lot of time thinking about the interface and trying to make it easy to use, but still powerful. We built a new notebook layout engine that allows you to run two queries and see the results side-by-side. Deleting a cell sends it to an archive, so you don't have to worry about losing data. Data visualizations are accessible with one click. And if you're just trying to query data on a CSV or JSON file, you can just drag and drop it onto the page.
One correction about Redash: Redash is open source and you can host it yourself too, in case you have an issue trusting a 3rd party with your data.
The proxy approach is something I thought about several times, but I am aiming at providing a collaborative tool, where it's easy to share the results and work with others. Depending on a proxy running on a user machine breaks this.
But having said that, I'm sure there are many whose needs Franchise will fit very well. So kudos on launching a great tool! :)
I'd like to confirm, we are hosting redash 2.0 (recently updated from 1.0) to our own server and use it as a query portal and collaboration tool to a number of databases (including mysql, postgres and oracle). Query sharing is really important there are users that don't know SQL but know how to run "peter-21" or "jim-32" with redash.
Everything is smooth till now thank you for the great product!
I've used superset and honestly, it kinda sucked as a user. Maybe it will thrive now it's not under AirBnB's ownership, but I more than welcome competition in this space.
This is super impressive, and small, too. This is a lot less code than I would have imagined for such an ambitious project. The JS ecosystem has gotten really strong in recent years in terms of library support and how concisely complex tasks can be implemented.
Would love to use it. But I get this error in Chrome when I try to connect:
> Disconnected from PostgreSQL (Error: Too many result rows to serialize: Try using a LIMIT statement.)
This is on my console (my credentials replaced with ellipses):
~> npx [email protected]
npx: installed 140 in 12.533s
franchise-client listening on port 14645
opened connection
received: {"action":"get_postgres_credentials","id":1}
received: {"action":"open","db":"postgres","credentials":{"id":1,"host":"localhost","user":"...","database":"...","port":"...","autofilled":true,"password":"..."},"id":2}
received: {"action":"exec","sql":"SELECT table_schema, table_name, column_name\n FROM information_schema.columns \n WHERE table_schema not in ('pg_catalog', 'information_schema', 'pg_internal')","id":3}
Error: Too many result rows to serialize: Try using a LIMIT statement.
at Object.query (~/.npm/_npx/736/lib/node_modules/franchise-client/response.js:128:23)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
I ran the SELECT query in Valentina Studio and got 11000+ rows in 4s. Maybe raise the acceptable row count to 100,000?
This looks really nice. Having a bit of trouble getting the bridge running on node v6.11 on Windows 10.
The app mentions requiring the latest version of node but it's not clear whether you need the standard LTS (6.11) or the bleeding edge (8.5).
Thanks!
> npx [email protected]
franchise-client\server.js:16
ws.on('message', async message => {
^^^^^
SyntaxError: missing ) after argument list
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:542:28)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:389:7)
at startup (bootstrap_node.js:149:9)
I've been meaning to learn SQL for a long time, but I'm unfortunately in an unrelated field. Could anyone recommend some resources for learning SQL? Preferable targeted towards an experienced programmer
"Experienced programmer" can be a hindrance to learning SQL, depending.
You will need to set aside all your imperative, procedural habits. To use SQL effectively you need to think in terms of sets, unions, intersections, and declarative statements.
If you find yourself thinking "row at a time" you might need to stop yourself. It's not always wrong, but a common mistake many programmers make when first learning SQL.
Khan academy has a great in-browser environment for learning. Unfortunately the lessons are given in a tone that seems to be targeted towards middle-schoolers, so it can feel a bit irritating at times.
What's your unrelated field? In my experience there's a way to shoehorn SQL into any application if you really want to. Sometimes even when you really don't want to.
This looks great! Support for Google sheets, other than export/import? I was recently surprised at how horrible an experience it is to script Google sheets - in that case for "survey" (questionnaire form) data - either locally with python or (shudder) the js/script functionality of Google sheets...
I just pulled up the crime stats example data: How do you tell it that a given table can be plotted on a map? I don't have the file with me at the moment, but I do have an SQLite database with coords and datetimes I'd love to plot.
Cool but it's hosted. I'd pay for a sql-notebook application that ran on OSX. I'm always working on speculative features or analysis but security, policy, and portability reasons I'd rather have a desktop app.
Like someone else said, this is hosted (but there's a way to run it yourself). But, I don't see any pricing. Is that something they plan on adding later. Or, how do they plan to keep it online over time?
Not OP - but you get that error in Microsoft Edge on windows 10. Demo works on Internet Explorer 11 on windows (but without map option using Crime in LA example) and in Edge all the demos appear to run, map and graphs get populated etc, then page looses the db connection.
We're hoping to work with the community to add support for new databases. If you open an issue at https://github.com/hvf/franchise/issues we can track Teradata progress there.
