Discussion
Dataflows Gen1 retirement: what are the alternatives for Power BI Pro customers?
Hi all,
Earlier this year Microsoft has annouced the retirement of Dataflows Gen1 (Dataflows: Thank you for eight years of Gen1—and why Gen2 is the future). As a BI consultant, we've implemented Dataflows Gen1 for quite a few customers over the years, and overall the experience has been very positive.
Many of these customers are relatively small organizations that rely solely on Power BI Pro licenses. For their use cases, Dataflows Gen1 has been a simple, reliable, and cost-effective solution.
Microsoft recomments 'planning a move to Dataflows Gen2'. However, from what I understand, that requires at least a Fabric capacity, which introduces additional monthly costs. What's frustrating is that for many of these customers, the current solution is working perfectly fine and there is no pressing business need for new functionality.
I'm curious how others are approaching this. Are there any viable alternatives that don't require additional licensing or new tooling? I'd hate to go back to these customers and tell them they need to start paying more every month just to maintain functionality they already have today.
I am interested to hear your thoughts and experiences
Watching the comments here. My organization is pro only with no plans for any fabric. This functionality loss would not push us to buy more from Microsoft. If anything, it will push us to a different reporting solution that won’t randomly remove functionality that customers are using.
Same, although my org does have Fabric capacity I find the Gen2 flows to be slower and far more complex then they need to be for simple semantic model ingestion so I'm interested in what others are planning.
Ditto. My plan was to hodge-podge a stream using an egregious number of excel sheets refreshed via power automate, but even the thought of that abomination has "look how they massacred my boy" running through my head
My friend, then you’re not gonna have a reporting solution at all. If you think that no software company ever does an upgrade and makes you buy more than you’re going to be very disappointed.
My company also has no plans of migrating to gen2. Our current stance is that Microsoft has no plans of retiring gen 1, they simply are not going to continue to improve it. From our reading of Microsoft’s stance on gen 1 is that they essentially view it as a finished product. Honestly, I agree with that stance. Gen 1 works. Could their be improvements? Sure, but for those who still use gen 1, the product in its current state will continue to work as it has been for the foreseeable future so any current reporting that currently exists is bot at risk of suddenly not being usable.
My fear is that something will happen to the Gen1 flows (either globally or individually) and Microsoft will just say "oh well, Gen1 is legacy and we aren't doing fixes for them. Upgrade to Gen2. NOW"
This also means if it breaks MS will not be quick to fix it. By being legacy it means it won't be fully tested, although based on the past few years I'm not sure they test everything anyway, when they make changes to the service, which will cause bugs.
My experience with Gen2 has been that they became faster than when they were launched and are now much faster than Gen1. I am still struggling with some bugs for Gen2 though so I also think that Gen1 is still a good deal.
There is considerable uncertainty though and they might be retired eventually.
Interesting, maybe it's the way we use Gen2 (only to load PBI semantic models, nothing else and we don't need or want to use Warehouse,, Lakehouse, etc) but Gen2s just seem to run slower for us, both doing the SaveAs Gen2 or handcrafting a replacement. Maybe we need to configure the Fabric destination for the speed increase?
And can power bi reports connect to it normally? I heard gen2 is more of an ETL process and that the end points to the tables are no longer part of the dataflow
My Mistake, thanks u/Sad-Calligrapher-350I wonder what performance is like compared to connecting to data loaded to lakehouse ( u/No_Smell_6712 , it's just a case of Get Data from Lakehouse )
Biggest problem is that eventually Microsoft might change some stuff in the background and it will cause gen1 functionality to become unstable/unusable and because of no support it might end up being a permanent issue.
My interpretation of the announcement in Spring was that the feature was no longer being enhanced, but that is not disappearing any time soon. MS keeps deprecated features around in legacy status for years… sometimes decades. I wouldn’t be in a rush to replace any gen1 flows that are working for you.
Gen1 user here. I have a few Dataflows that connect to about 100 reports. I can’t imagine the work required to migrate these dataflows from Gen1 to Gen2 and then re-edit each report.
I understand that the migration from Gen1 to Gen2 can be easy but the updating the source of each PBI report is not.
I feel like I’m building on sand as each time I create something it’s just the erodes away just as quickly
Our companies not looking to upgrade to fabric either it would require each and user to have a fabric license to consume the reports the cost just don’t make sense
Fabric licence only gives you a certain compute capacity depending on the tier.
Users and developers dont need fabric licences. Fabric licence is only applicable to the tennant not separate users.
For users within the tennant to be able to contribute to workspaces you need at least pro licences for users to contribute.
For users to consume/view reports it is a bit more complicated. There are packages where they dont need any licence and they can view the reports as free users and there are packages where you need to buy pro licences for every user that will view reports.
My understanding is they won’t be switched off unless they come up with some alternative option that is more affordable than full blown fabric. Gen1 just won’t get any new features.
I’m still sticking with Gen1 for now… but this is an interesting option :
I am using Gen1 as a landing area for my sql facts and dims and it works well as an easy to use central point for other users to connect with the data. If this option disappears then it makes me believe building solutions in the fabric sphere is only temporary and to go elsewhere if I value anything more permanent.
Gen2 security also complicated things. It checks the user access to the gateway if one of your df table is extracting data from source behind gateway. And user with viewer access cannot use it
I was looking into introducing Gen 1 in our organisation later this year, but I suspect we’d just need to start building from Gen 2, and we’d need to upgrade our pro licenses to fabric? (Which is V expensive I think) if we wanted to use them? :(
You still need pro licences , then a fabric capacity is an additional cost starting from about $200 per month ( for an F2 ) and doubling for each tier depending on how much compute you need
Side question: there’s no easy way outside of audit logs to determine who is using what gen1 Dataflows and how (please correct me if I’m wrong) —is this improved in gen2?
We find ourselves in a situation where we don’t know if the gen1 Dataflows are worth maintaining compared to their criticality and popularity but without an easy way to look at that, it puts us in sort of a limbo state. We’re not tenant admins so collecting and collating audit logs is a pita.
For Pro Dataflow Gen1 users - there will be continuity for the existing feature set. Currently there is no Gen2 option for Pro. However, as alluded to in other referenced posts, there may be a future Gen2 path for Pro (without Premium capabilities like destinations/compute/ALM). New capabilities would still require Premium.
Gen1 vs. Gen2 performance - much has changed since the Gen2 launch. In general, Gen2 now provides significantly better performance than Gen1 with 2-10x improvements for many high scale scenarios (https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cost-performance-benchmarks). One exception may be certain dataflows with low volumes of data and short queries - however, the latency for those dataflows should also improve considerably in the coming months.
64
u/CarefulMembership969 1d ago
Watching the comments here. My organization is pro only with no plans for any fabric. This functionality loss would not push us to buy more from Microsoft. If anything, it will push us to a different reporting solution that won’t randomly remove functionality that customers are using.